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ABSTRACT 


Part  I  Is  devoted  to  the  general  theory  of  digital  filters. 

The  filtering  theories  for  both  continuous -tine  and  discrete-time  sig¬ 
nals  are  formulated  In  terms  of  abstract  Hilbert  space,  with  the  notion 
of  a  stable  filter  defined  as  a  boimded  linear  operator.  This  abstract 
settlxig  allows  the  z-transfozm  to  be  defined  with  the  same  generality  as 
the  Fourier  transform.  A  specific  Isoaorphlsm  Is  then  constructed  which 
connects  the  filtering  theories  for  continuous -time  and  discrete-time 
signals,  and  In  the  linear  time -invariant  case  the  two  theories  are  shown 
to  be  essentially  Identical.  This  means  that  many  optimization  problems 
can  be  solved  simultaneously’  for  continuous -time  and  digital  systems. 

In  the  second  part,  the  Isomorphism  developed  in  Part  I  is  used 
to  reduce  the  approximation  problem  for  digital  filters  to  that  for 
continuous -time  filters.  This  allows  the  designer  of  digital  filtering 
computer  programs  to  use  many  of  the  concepts  which  have  proven  Important 
to  the  communications  engineer. 

In  the  last  part,  the  problem  of  estimating  the  power- spectral- 
density  of  a  signal  from  equally  spaced  samples  Is  discussed.  It  Is 
shown  that  bandpass  digital  filters  generate  a  class  of  spectral  windows 
which  produce  always  jxjsitlve  estimates  of  the  power-spectral-density. 

The  optimum  bandwidth  and  shape  of  such  a  filter  are  then  derived. 

Finally,  a  method  for  identifying  unlmown  parameters  In  the  power-spectral- 
denslty  of  a  digital  signal  Is  presented. 
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PREFACE 


Historically,  nethods  for  process iiig  signals  that  are  functions 
of  continuous  time  were  developed  long  before  the  advent  of  high  speed 
digital  computers.  When  high  speed  cozoputlng  facilities  did  become 
available,  the  communications  and  control  engineers  were  not  the  people 
who  developed  canqputing  techniques.  As  a  result,  the  filtering  theory 
that  had  been  highly  developed  for  contlnuous-tlae  signals  was  not  applied 
In  full  force  for  the  processing  of  digital  signals. 

The  main  purpose  of  this  thesis  is  to  tie  together  the  theories 
of  filtering  digital  and  analog  Information.  This  will  enable  the  data 
analyst  to  carry  over  effectively  to  his  domain  many  of  the  concepts  which 
have  been  Ir^jortant  to  network  designers.  In  particular,  all  the  approxi¬ 
mation  techniques  developed  for  continuous -time  filters  become  available 
for  digital  applications. 

The  strong  link  that  is  developed  between  the  digital  and  con¬ 
tinuous  domains  will  also  be  of  theoretical  value.  It  will  present  to  us 
a  unified  picture  of  signed  and  filtering  theory,  a  picture  chat  is 
equally  applicable  to  digital  and  continuous  signals. 
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PART  L:  THE  GINER\L  THEORY  OF  DIGITAL  FILTERS 


1.  Introduction 

It  Is  easy  to  observe  a  parallel  between  signal  theory  for 
signals  with  a  continuous  time  parameter  and  signal  theory  for  discrete- 
time  signals.  In  factj  It  Is  common  practice  to  develop  In  detail  a 
filtering  theory  for  continuous- time  signals  and  to  pay  less  attention 
to  the  discrete  theory,  with  the  assumption  that  the  derivation  in  the 
discrete  case  follows  the  one  for  continuous- time  signals  without  much 

TOO 

change.  Thus,  without  going  into  details,  '  the  Wiener  filter  for 
a  noise- corrupted  continuous- time  signal  is 


^  “  ^r^riC®)  i 


and  the  optimum  filter  in  the  discrete  ceise  is 


°  YU)  [  Y(z)'^  ]lK|.|<  ' 


where  r  is  the  uncorrupted  signal  and  rj  is  the  corrupted  signal.  On 
the  other  hand,  the  two  cases  are  always  considered  as  distinct  and  ' 
essentially  different  situations. 

This  correspondence  between  continuous  and  discrete  phenomena 
is  far  from  accidental,  howe''  jr.  In  fact,  when  both  theories  are 
axiomatized  in  terms  of  Hilbert  space  theory  (Lp  and  1^  theory),  they 
are  isomorphie.  This  simple  fact  is  quite  illuminating  and  leads  to  a 
more  unified  theory  of  filtering  and  prediction. 
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Usually,  It  Is  assumed  that  the  signals  of  Interest  are  of 
exponential  order  as  t  becomes  Infinite,  l^ls  leads  to  tvo-slded 
Laplace  trauisforms  which  converge  In  a  strip  In  the  s-plane,  or  double 
ended  z-transforms  which  converge  In  an  annulus  of  the  z-plane.  Ihls 
Is  replaced  in  Hilbert  space  theory  by  mean  convergence  on  the  jee-axis 
and  unit  circle,  respectively.  In  one  sense  the  signal  spaces  and 
Ig  are  more  restrictive,  because  they  do  not  include  signals  of  posi¬ 
tive  exponential  order.  On  the  other  hand,  assuming  that  ve  are 
dealing  with  physically  real  signals,  the  spaces  1^  and  Ig  are  more 
general  and  intuitively  satisfying;  roughly,  they  include  all  signals 
whose  total  energy  content  j  j  finite. 

Our  main  purpose  in  this  first  pairt,  then,  will  be  to  imbed 
the  theory  of  continuous- time  signals  in  U,  theory  and  the  theory  of 
discrete-time  signals  in  Ig  theory;  and  to  show  that  the  filtering 
theories  for  these  two  classes  of  signals  are  essentially  the  same. 

We  will  thus  arrive  at  a  definition  of  digital  filter  that  is  as 
general  as  the  definition  of  continuous-wave  filter,  and  we  will  show 
that  many  problmns  in  the  design  of  discrete- time  systems  need  not  be 
re-solved.  As  a  by-product,  we  will  see  how  well  Hilbert  space  theory 
is  suited  to  describe  linear  filtering  theory  for  both  continuous  and 
discrete  time. 

While  Youla,  Castrlota  and  Carlin,  and  other  network  theo¬ 


rists  have  applied  Lg  theory  to  continuous- time  network  theory,  to 
the  author's  Imowledge  lg  theory  has  not  been  applied  to  the  z- transform 


k 


and  the  Isomorphism  between  and  Ig  has  not  been  exploited  by 
electrical  engineers. 

We  begin  with  a  review  of  the  elements  of  Hilbert  space 

theory. 

2.  A  Review  of  Hilbert  Space  Theory 

We  will  adopt  the  widely  accepted  definition  of  abstract 
Hilbert  space.  That  is:  a  set  H  of  arbitrary  elements  (some¬ 

times  called  functions  or  vectors)  is  termed  a  Hilbert  space  if: 

I.  H  is  a  linear  space. 

II.  An  inner  product  is  defined  in  H  as  follows:  to  every 

pair  of  elements  f,g  there  is  associated  a  complex  number 
(f,g)  such  that 

1)  (f,g)  =  TsTi 

2)  (Ctf,g)  =  a(f,g) 

3)  (fi-*-f8,g)  =  (fi,g)  +  (fp,g) 

4)  (f,f)  =  0  if  and  only  if  f  =  0. 

III.  Tae  space  H  is  complete  in  the  metric  Ilf-gll  = 

IV.  H  is  infinite  dimensional;  tliat  is,  for  any  integer  n 
there  are  n  linearly  independent  elements  in  H. 

V.  H  is  separable;  that  is,  H  contains  a  countable  and  dense 
set.  (This  condition  is  often  omitted,  allowing  spaces 
of  dimension  higher  thon^o)* 
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!nius,  a  Hilbert  space  is  a  complete,  separable,  infinite* dimexisional 
Euclidean  space. 

Historically,  two  concrete  realizations  of  Hilbert  space  play 
central  roles.  The  first  is  the  space  Lg(a,b),  which  is  defined  to  be 
the  set  of  all  complex- valued  Lebesgue  measurable  functions  on  (a,b) 
such  that 

b 

lf(t)l^  dt  <  00 
a 

The  inner  product  in  this  space  is  defined  by 


(f>s)  = 


f(t)g(t)  dt 


a 

Two  functions  in  Lg  are  considered  equal  if  they  differ  only  on  a  set 

/ 

of  measure  zero.  Since  the  metric  in  this  space  is  (f-3,f-s)~/  the 
sequence  f^  will  approach  f  if 


b 

r 

lim  I  'fn-fP  dt  =  0  . 

n  -  00  J 
a 

This  will  be  called  mean  convergence  and  will  be  written 


f  =  l.i.m.  fn  . 
n  -  00 


The  other  Hilbert  space  is  called  lg.  It  is  defined  to  be  the 


set  of  all  sequences  of  complex  numbers 
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X 


lxi,X2 


f^n, 


] 


Batisfying  the  condition 

00 

|xnP  <  00  . 

n=l 

Here,  the  inner  product  is  defined  by 

00 

(x,y)  =  ^  Xn  . 

n=l 

(Sometimes  it  will  be  convenient  to  think  of  Ig  as  containing  double 
ended  sequences:  {.,.x-i,  Xq,  Xi,Xg,...}.  The  theory  is  really  the 
same. ) 

For  us,  the  space  Lg(-oo,  loo)  will  play  the  role  of  the  space 
of  continuous  time  signals,  ar-^  Ig  will  represent  the  space  of  discrete- 
time  signals. 

An  isomorphism  frcwn  one  Hilbert  space  Hj  to  another  Hilbert 
space  Hg  is  a  one-to-one  linear  transformation  U  from  onto  Ilg  such 

that  (lhc,Uy)  =  (x,y)  for  every  pair  of  vectors  x.y  in  .  /ji  isomorphism 

preserves  all  the  structure  embodied  in  the  definition  of  Hilbert  space 
and  isomorphic  Hilbert  spaces  are  geometrically  indistinguishable  and 
for  our  purposes  can  be  considered  as  identical. 

The  following  theorem  is  central  for  our  purposes: 


Theorem  1-  All  Hilbert  spaces  are  isomorphic. 
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The  proof  of  this  theorem  is  interesting  and  useful.  We  now  review  its 
main  points. 

1.  Since  H  is  separable,  we  can  choose  in  H  a  countable  dense  set. 
From  this  set  we  can  construct  an  orthonormal  set  {hx,hp,h3,. . .  ]  that  is 
complete  in  H.  That  is, 


(hi,hj) 


0  is  i/j 
.1  if  i=J 


and  linear  combinations  of  the  are  dense  in  H. 


2.  Tnis  implies  that  any  element  of  H  can  be  approximated  with 
arbitrary  accuracy  by  linear  combinations  of  the  hi.  If  we  define  the 
partial  sum  of  a  generalized  Fourier  series  by 

n 

Sn  “  ^  f 

k=l 

then  the  distance  between  Sn  and  f  in  tlie  metric  of  H  is  smallest  when 


cj^  =  (f,h^) 

In  that  case,  we  have  in  fact 

n 

llf-s„||=  =  (f,f)  -  I 

k=.l 


Now  let  n  approach  infinity.  Since  Sj^  is  the  best  n-th  order  approxi¬ 
mation  to  f,  and  since  the  orthonormal  set  {hi,h2,...}  is  complete,  we 


must  have 


e 


llm  I  |f-Snl  1=  =  0 

n  -»  00 


and  hence 


oo 


(f,f)  =  I  |C1;|= 


k=l 


3.  Conversely,  let  c^jCg,...  be  a  sequence  of  numbers  such  that 


I  IckP  <  “ 

k=l 


and  construct  the  sequence  of  partial  sums 


n 

^‘n  =  ^  ckhk 
k=l 


It  then  follows  that 

n+p 

llW^nir-  y  IckP 

k=n+l 


As  n  approaches  infinity  the  richt  side  goes  to  zero.  The  left  side 
must  go  to  zero,  and  this  implies  that  the  sequence  f^  is  fundamental. 
The  fact  that  H  is  complete  in  its  metric  then  implies  that  there  is  a 
limit  function  f  c  H  such  that 


llf-fnii  -  0 

as  n  -  00.  It  then  follows  easily  that 


cj^  =  (f,hj,) 
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and  that 

00 

(f.f)  -  £  Ukl'  • 
lA 

4.  We  now  assign  to  each  element  in  H  the  sequence  [ci^Cg,...]  of 
its  Fourier  coefficients.  By  step  2  above  this  is  an  element  in  Ig. 
Furthermore,  by  step  3,  for  each  element  {ci,Cg,...}  in  1^  there  is  an 
f  in  H  which  has  Fourier  coefficients  {ci,C2,...}.  This  coirespondence 
is  linear,  one-to-one,  onto,  and  preserv'es  norm.  It  is  therefore  an 
isomorphism,  and  we  have  therefore  shown  that  any  Hilbert  space  is  iso¬ 
morphic  to  Ig ,  and  hence  to  any  other  Hilbert  space.  In  the  case 
H  =  L2(a,b)  this  procedure  corresponds  to  mapping  a  function  to  the 
sequence  of  its  coefficients  in  some  orthogonal  expansion  on  the  inter¬ 
val  (a,b);  such  as  an  ordinary  Fourier  series  on  (0,2j0  or  a  Laguerre 
series  on  (0,oo),  for  example. 

\/ith  this  review  we  go  on  to  apply  these  ideas  to  more  familiar 
situations . 

3«  /jciomatization  of  Deterministic  Signal  Theory 

In  most  deterministic  situations  encountered  by  engineers,  the 
signals  are  either  functions  of  a  continuous  tine  variable  or  a  discrete 
time  variable.  In  either  case,  the  total  energy  contained  in  a  signal 
is  really  finite,  even  though  we  make  up  models  which  deny  this.  For 
example,  we  say  that  a  step  input  is  applied  to  some  system  at  t  --  0  and 


we  write 
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This  is  clearly  not  realistic.  The  definition 


,0  -00  <  t  <  0 

f(t)  =  \l  0  <  t  <  T 
0  T  <  t  <  00 


where  a  is  very  small,  describe  the  situation  Just  as  well.  Thus, 
without  serious  limitation,  we  can  assume  that  any  wave  will  have  a 
finite  total  energy.  With  this  assumption,  Hilbert  space  L,{-co,oo), 
with  its  convenient  completeness  and  with  its  continuous  Fourier  trans¬ 
form,  provides  a  neat  setting  for  our  discussion  of  deterministic 
signals  which  are  functions  of  the  continuous  time  parameter  t. 

Similarly,  when  a  signal  is  a  function  of  discrete  times,  the 
Hilbert  space  1^  is  a  realistic  model  with  many  convenient  mathematical 
properties.  From  now  on,  a  function  in  Ln (-00,00)  will  be  called  an 
euialog  signal,  and  a  function  in  1^  will  be  called  a  digital  signal. 

It  is  now  rather  startling  and  counterintuitive  to  the  engineer 
that  L3(-oo,oo)  and  L,  are  isomorphic.  tSter  all,  any  signal  in  Ip 
could  have  been  obtained  by  sampling  at  discrete  times  any  one  of  an 
infinite  number  of  analog  signals.  The  problem  here  is  that  the  mapping 
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Lg (-00,00)  -•  Ip  defined  by  sampling: 

f(t)  -  {...,f(-2T),f(-T),f(0),f(T),...} 

is  not  an  isomorphism,  since  it  is  not  one-to-one.  Nevertheless,  Lp 
can  be  made  isomorphic  to  Ip  by  an  appropriate  choice  of  mapping;;  in 
the  same  way,  for  example,  that  the  Abelian  group  of  integers  can  be 
made  isomorphic  to  the  Abelian  group  of  even  integers. 

k.  The  Transform  Domains 

Our  next  goal  will  be  to  construct  a  specific  isomorphism 
which  can  serve  as  a  concrete  link  between  the  amlog  and  digital  sig¬ 
nal  spaces.  Naturally,  we  would  like  the  mapping  to  have  some  intuitive 
significance.  The  very  natural  correspondence  provided  by  sampling 
analog  signals  lias  been  ruled  out  because  it  is  not  an  isonorphism.  It 
would  still  be  desirable,  however,  to  have  the  left  half  s-plane  corre¬ 
spond  to  the  interior  of  the  unit  circle  in  the  z-plane,  because  these 
regions  seem  to  play  analogous  roles,  even  "vdien  no  signs  Is  have  been 
sampled.  To  make  these  ideas  precise,  we  must  add  the  Laplace  transform 
and  the  z- transform  to  our  Hilbert  space  theory. 

The  key  theorem  for  the  construction  of  a  transform  domain  for 
Lp(-oo,oo)  is  called  Plancherel's  Theorem:®*^ 


Theorem  2,  (Plancherel)  If  f(t)  e  Lp (-00,00),  then 
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F(q) 


1. l.m. 

-  00 


A 

f  f(t)e"'^^  dt 


-A 


exists  for  s  -  j(ij,  and  F(jco)  e  Lp(-oo,oo;. 
Furthermore, 


+00 


(f,f)  =  [  !f(t)| 


iCX) 


J 


dt  = 


idjij 


1f(s)|'’  ds 


-oo 


-joo 


and 


(I-l) 


(1-2) 


f(t) 


l.l.m. 

A  -  00  u 

-JA 


F(s)e®^  dt 


(1-3) 


Analytic  extension  of  F(jtD)  to  the  rest  of  the  s-plane  (via  (I-l)  vhen 
it  exists,  for  example)  will  give  us  the  Laplace  transform. 

V/o  will  also  use  Parseval's  Theorem: 


Theorem  3 »  ( Pars eval )  If  f,g  e  L, (-00,00),  then 


00 


r 

:f,s)  '  '  f(t)e(t 


)  dt  = 


2nj 


-00 


jP 


-Joo 


F(s)G(-s)  ds 


(1-4) 


Tile  tiieory  required  for  the  analogous  construction  of  a  z- 
transform  for  digital  signals  is  really  no  more  than  the  theory  of 
Fourier  series.  Tnink  of  the  original  periodic  function  as  the  2- 
transform  evaluated  on  the  unit  circle  in  the  z-plane;  and  thinlc  of 
the  Fourier  coefficients  as  the  values  of  our  digital  signal.  The 
Riesz- Fischer  Iheorem”'  then  reads: 
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Theorem  U.  (F.  Riesz- Fischer)  If  {fn}^  ..  c  I-?  ohen 

.4  *“  ^”3 


iz) 


1. 1  .m. 

N  -•  CO 


I'l 


n=-N 


(1-5) 


exists  for  z  =  ,  and  F(e'^^i^)  c  L2(0,2:i/T),  vhere  (£  is  the  independ¬ 

ent  variable  of  L,(0,2n/T),  and  this  w  is  unrelated  to  the  co  used  in  the 
s- plane. 

Furthermore, 

+00  r 


^  |fnP 

n=-aD 


|r(z)  p 

2rO  J  2 

|2  =1 


(1-6) 


and 


;  F(z)z’^  .  (1-7) 

z 

1=1 

As  in  the  analog  case,  the  analytic  extension  of  F(e'^—  )  to  the  rest  of 
the  z-plane  vill  coincide  with  the  ordinary  z-transform,  which  is 
usually  defined  only  for  digital  sigimls  of  exponential  order. 

Parseval's  relation  also  holds: 


Tneoren  (Parseval)  If  ^  then 


+00 

([fnl.fcni)  =  y 


lb  F{2)0(z-M  -42. 


n=-oo 


z 


.  (1-8) 


Ih 


To  suinnarize,  we  have  defined  an  analog  signal  space  L,(-oo,oo); 
together  with  its  transform  domain,  which,  when  s  =  juj,  is  also 
L^(-oo,oo).  Analogously,  we  have  defined  a  digital  signal  space  Ip} 
together  with  its  transfom  domain,  which,  when  z  =  e*^-^,  is  L.p(0,2u/T). 
We  now  are  in  a  position  to  define  a  specific  isomorphism  between  the 
analog  and  digital  signal  spaces  via  their  transform  domains;  a  procedure 
which  was  hinted  at  before. 

5»  A  Specific  Isomorphism,  p 

Remembering  that  we  wish  to  map  the  jco-axis  in  the  s- plane  onto 
the  unit  circle  in  the  z-plane,  the  familiar  bilinear  transformation 

^  -  2-1  „  _  1+s 

z+1  1-s 

is  a  natural  choice.  There  is  an  additional  factor  required  so  that  the 
transformation  will  preserve  norms.  The  image  {£n}  e  1,  corresponding 
to  f(t)  €  La(-oo,oo)  will  then  be  defined  as  the;  sequence  with  the  z- 
transform 

F(z)  =  —  F  f-  -  '  -  ,  where  we  use  the  underscore 

z+1  z+1  '  to  denote  digital  domains. 

Thus,  the  mapping  (i;L„  (-oo  ,oo )  1„  is  defined  by  a  chain  which  goes 

from  L.p(-oo,oo)  to  I*,(-oo,co)  to  L,(0,2rv/T)  to  1^,  as  follovrs: 
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u:f(t)  -  F(s) 


(1-9) 


The  inverse  mapping  is  easily  defined,  since  each  of  these 
steps  is  uniquely  reversible: 

-  F(2)  -  F  F(®)  -  * 

“  1-s  "  ^  1-3 


The  mapping  and  Its  relations  to  the  various  spaces  are  shown 
schematically  in  Figure  1. 

To  show  that  is  indeed  an  isomorphism,  we  first  verify  that 

preserves  the  inner  product.  Let  f  end  g  be  any  two  analog  signals. 

By  theorem  3  (Parseval’s  relation  for  analog  sigiials),  we  have 

joo 

r» 


(f#g)  = 


2nj 


F(s)G(-s)  ds 


-Jos 


Letting  z  = 


1+3 
1-s  ' 


this  becomes,  with  some  algebraic  manipulation, 


(-^g)  = 


(t;  P(z)G(: 


z  =1 


Then,  using  Iheorem  5  (Parseval's  relation  for  digital  signals),  we 
find  that  u  does  preserve  the  inner  product: 


|i  is  obviously  linear  and  onto.  We  can  now  show  that  is  one-to-one 
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in  the  following  vay:  if  f  /  g,  then  (f-c,f-g)  =  ( ^ 

/  0;  which  implieo  that  hence  that  ^  is  one-to-one. 

Ihis  establishes  the  fact  that  ^  is  an  isomorphism. 

We  note  here  that  under  the  isomorphisms  u  and  functions 
with  ra-tional  transforms  are  always  matched  with  functions  with  rational 
transforms,  this  fact  following  from  the  nature  of  the  transformation  u. 
Ihis  is  a  great  convenience,  since  many  of  the  functions  commonly  en¬ 
countered  in  engineering  problems  have  transforms  which  are  rational 
functions  of  s  or  z. 


6.  The  Orthonormal  Etcpansion  Attached  to  u 


In  our  review  of  Hilbert  space  theory  we  showed  how  a  set  of 
orthonormal  functions  generated  an  isomorphism  between  two  Hilbert 
spaces.  It  should  come  as  no  surprise,  then,  to  learn  that  the  iso¬ 
morphism  u  could  have  been  so  generated.  This  section  will  be  devoted 
to  finding  this  orthonormal  expansion. 

V/e  start  with  the  z- transform  of  the  digital  signal  which 
is  the  image  under  u  of  an  arbitrary  analog  signal  f(t): 


00 


n=-oo 


By  (l-7)f  the  formula  for  the  inverse  z- transform,  we  have 


O  Ji—  F  z" 

Z+1  '  z+1  ^ 


dz 


UI=1 
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Letting  z  =  ■j'  this  integral  becomes 

i“  s 


In  = 


^  I  F(3) . 

2TtJ  J  1+3  1-s  ^ 

-Joo 


ds 


(I-ll) 


By  Parseval's  relation  (l-U),  this  can  be  rewritten  in  terras  of  time 
functions  as 


fn  =  f(t)  \n(t)  dt  , 


(1-12) 


where  the  Xn('t)  given  by  the  inverse  Laplace  transform  of  the  factor 
appearing  in  the  integrand  of  (I- 11)  with  s  replaced  by  -s.  Thus; 


We  see  immediately  that,  depending  on  whether  n  >  0  or  n  <  0, 
vanishes  for  negative  time  or  positive  tine,  respectively.  By  mani¬ 
pulating  a  standard  transform  pair  involving  Laguerre  polynomials,  we 


find: 


e''^  Ln-i(2t)  u(t)  ,  n  =  1,2,3,..., 

=1  n  r  t 

‘JZ  L_^(-2t)  u(-t)  ,  n  -  0,-l,-2,..., 

(1-14) 

where  u(t)  is  the  Heaviside  unit  step  function,  and  Lj^(t)  is  the  Laguerre 
polynomial  of  degree  n,  defined  by; 


Ln(t)  = 
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The  set  of  functions  ^  complete,  orthonomal  sot  on  (0,oo), 

and  are  called  Laguerre  functions.  They  have  been  employed  by  Lee,^^ 

12 

Wiener,  and  others  for  netvori:  synthesis;  and  arc  tabulated  in 
V/iener,  and,  vith  a  slightly  different  normalization,  in  Head  and 
Wilson.  The  functions  similarly  complete  and  orthonormal 

on  (-00,0),  so  that  the  orthonormal  expansion  of  f(t)  corresponding  to 
(1-12)  is 

CD 

l(t)  =  ^  fj;>  Xpj(t;  .  (l”!^) 

n=-oo 

We  see  then,  that  the  values  of  the  digital  signal  for  a  0  correspond 
to  the  coefficients  in  the  Laguerre  expansion  of  f(t)  for  positive  t; 
and  that  the  values  of  the  digital  signal  for  n  <  0  correspond  to  the 
coefficients  in  the  Laguerre  expansion  of  f(t)  for  negative  t. 

Tnere  follows  from  this  representation  the  facu  that  the  iso¬ 
morphism  p  matches  one-sided  functions  with  one-"'  ■  .  f’unctions.  Tlaat 
is,  f(t)  =  0  for  t  <  0  if  emd  only  if  {fj^}  =  0  for  n  <  0;  and  similarly, 
f(t)  =  0  for  t  -•  0  i only  if  {f^^}  =  0  for  n  0. 

.lOr  orthonomal  expansions,  such  as  the  Heraite,  for  example, 
also  generate  isomorphisms;  but  these  will  not  be  as  convenient  and 
as  simple  for  our  purposes  as  the  Laguerre  expansion.  In  particular, 
Kautz,^^  Gabor, Huggins, and  others  have  considered  the  construction 
of  orthonomal  f -unctions  for  signal  representation. 

The  fact  that  the  mapping  u  is  eq-ui valent  to  a  Laguerre  expan¬ 
sion  can  sometimes  lead  to  a  q-aick  way  of  expanding  a  given  time  function 
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in  a  Laguerre  series.  One  need  only  find  F(s)  from  the  Laplace  trans¬ 
form  of  f(t)  and  expand  this  in  a  power  scries  in  z.  To  illustrate 
this,  and  the  way  that  the  mapping  n  works  in  general,  consider  the 
function 

i(t)  -  e“^  sin  t  u(t) 

This  function  is  in  Lp  and  its  Laplace  transform  is  ajialytic  in  the 
half- plane  Re(s)  -1.  Thus, 


and 


F(s) 

F(z) 


1 

(s^)^  +  1 

nTe  (z-H) 

+  2z  +  1 


.,-4 


Th-is,  by  (1-15), 


f(t)  =  ^2  Xl(t)  + 


29 


11 


125 


\,(t) 


625 


These  coefficients  can  be  checl:ed  by  carrying  through  the  integrations 
indicated  in  (1-13). 


7.  Stable  Filters  as  Bounded  Linear  Opeiators 

We  cone  now  to  the  problem  of  incorporati.ng  within  our  frame¬ 
work  the  concept  of  "filter"  or  "transfer  function."  That  is,  we  wish 
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to  formulize  the  notion  of  a  device  vhich  transforms  one  element  of 
Hilbert  space  into  another.  Guch  a  device  might  be  a  network  of  resis¬ 
tors,  inductors,  and  capacitors  which  transforms  one  analog  signal  into 
another;  or  it  night  be  a  digital  computer  which  transforms  one  digital 
signal  into  another  such  signal.  We  assume,  mostly  because  we  must  to 
achieve  any  generality,  that  such  filters  are  linear.  It  is  also 
reasonable  to  expec  t  that  if  we  limit  the  energy  content  of  the  input 
function  to  a  stable  filter,  that  the  energy  content  of  the  output  will 
be  limited. 

Fortunately,  operators  with  sucli  properties  have  been  studied 
widely  in  connection  with  Hilbert  space. /.n  operator  A  in  a 
Hilbert  space  H  is  defined  as  a  transformation  whicii  attaches  to  each 
element  f  in  H  some  element  Af  which  is  also  in  H.  f\n  operator  L  is 
said  to  be  li.iear  if 

.A(af  +  pg)  =  oAf  +  (3/^ 

for  any  f,g  in  II  and  any  complex  numbers  CL  and  p>.  Lastly,  correspond¬ 
ing  to  our  energy  requirement,  a  linear  operator  is  said  to  be  bo’.mded 
if  there  is  a  positive  real  n-umber  M  such  that 

llAfll  <  M  llfll 

for  all  f  in  H.  Tne  noitn  of  the  linear  operator  A  is  the  infimLun  of 
all  such  values  of  M,  and  is  written  I  jA  j  ] .  Equivalently,  the  nom.i  of 


A  can  be  defined  as 
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I  l-'^l  I  "  sup  I  j-|  .  (I-l6) 

TgH  M-1 

One  example  of  a  bounded  linear  operator  is  the  Fourier  trans¬ 
form.  By  (1-2)  tnis  operator  has  a  norm  equal  to  nTi/Sj. .  As  another 
example,  consider  a  simple  low-pass  RC  section  with  the  transfer  function. 


T 


If  an  input  wave  f(t)  is  applied  to  this  network,  the  total  ener,;;/  in 
the  output  will  be 


1 

2i.  j 


j 


00 


1f(s)P 


k 

- T 

<jS'  +  T 


ds  < 


I  1f(s)|=  ds 


-joo  -joo 

so  that  the  norm  of  this  operator  cannot  exceed  1.  Since  this  is  a 
passive  network,  it  is  to  be  expected  that  the  total  output  encri^y 
cannot  exceed  the  input  energy. 

We  are  thus  led  to  adopt  the  following  terminology:  A  bounded 
linear  operator  on  the  space  will  be  called  a  (linear)  analog  filter 
and  a  bounded  linear  operator  on  1,  will  be  called  a  (linear)  digital 
filter. 


It  is  now  a  direct  consequence  of  our  axiomatic  setup  that  any 
bounded  linear  operator  is  continuous  in  the  metric  of  Hilbert  space. 

r  ->00 

rnat  is,  if  ifnjn=l  is  a  sequence  of  functions  in  the  Hilbert  space  H, 
and  if  f  is  a  function  in  K  such  that 
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llm  llf„  -  f  II  -  0 

n  -  00 

then 

Urn  !  |/,f„  -  /‘f  I  I  -  0  , 

n  -  00 

vhere  A  Is  a  bounded  linear  operatoi'.  Tiiis  follows  immediately  fror’ 
the  fact  that 

l|;.fn  -  Afll  <  I!a||  •  !|f„  -  f||  . 

Continiiity  is  a  desirable  property  of  operators.  In  ,  for 
Instance,  it  means  that  if  input  functions  to  an  analog  filter  A 
approach  a  function  f  in  the  mean,  then  the  output  will  approach  t'S  in 
the  mean.  This  convenience  is  bought  at  the  price  of  considering  only 
functions  in  and  using  mean  convergence  as  the  convergence  criterion. 
If  we  Insist  on  thinking  in  terms  of  pointwise  convergence,  for  instance, 
we  lose  continuity;  as  the  following  example  shows:  Let  a  set  of  input 
functions  to  some  network  approach  the  delta  function.  The  pointwise 
limit  of  the  input  functions  is  then  0  almost  everywhere.  But  in  gen¬ 
eral  the  output  will  not  approach  0,  so  that  filters  will  not  be  con¬ 
tinuous  in  this  framework.  In  a  way,  our  convergence  criterion  is  more 
natural  than  pointwise  convergence:  for  a  sequence  to  approach  f  in 
the  mean  we  demand  only  that  the  total  energy  of  f^-f  approach  zero. 

Cince  u  can  be  thought  of  as  a  bounded  linear  operator  in  the 
abstract  Hilbert  space  H,  p  is  continuous.  Similarly,  the  Fourier 
transform  is  continuous.  Also,  since  1.,  and  Lp(0,2n/T)  are  isomorphic 
Hilbert  spaces,  the  z- transform  as  defined  in  Theorem  4  is  also  contin¬ 
uous.  V/e  summarize  these  facts  in  the  following  theorem: 
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Theorem  6.  All  bounded  linear  operators  are  continuous  in  the  metric 
of  Hilbert  space.  In  particular,  the  following  bounded  linear  opera¬ 
tors  are  continuous: 

1.  Analog  filters 

2.  Digital  filters 

3.  The  Fourier  transform 

U.  The  z- transform 

5.  The  isomorphism  u 

8.  The  Mapping  m  for  Filters  ond  the  Transforms  of  Filters 

Since  our  signal  spaces  are  new  equipped  with  operators,  it  is 
natural  to  extend  our  isomorphism  u  so  that  it  matches  operators  that 
act  equivalently  in  the  two  spaces  and  1;,.  More  precisely,  if  A  is 
an  analog  filter,  we  define  its  image  m(A)  -  .j  in  the  following  way: 
let  X  be  any  digital  signal.  Then  there  corresponds  to  x  a  unique 
analog  signal  n"^(x).  The  result  of  operating  on  this  analog  signal  by 
the  analog  filter  A,  Au"^(x),  is  also  well  defined.  This  new  aruilog 
signal  can  then  be  mapped  by  n  into  a  unique  digital  signal  uA^  ^(x), 
which  we  designate  as  the  result  of  operating  by  A  on  x.  Thus,  we 
define  A  to  be  the  composite  operator 

A  =  (l-l?) 

To  avoid  confusion  between  digital  filters  and  z- transforms  of  digital 
signals,  we  use  the  double  underscore. 


2h 


It  is  easy  to  see  that  the  mapping  H  for  operators  is  linear, 
one-to-one,  ajid  onto.  Given  a  di^iiital  filter  A,  its  corresponding 
analog  filter  is  To  show  that  the  norm  of  an  operator  is  pre¬ 

served  under  the  matching  u,  we  need  only  carry  out  the  following 
calculation: 


sup 

xela 


1 1^1 1 


sup 

xcl^ 


|p(Au'^ (x)  )|  1 


(1-18) 


=  sup 
xelp 


sup 
X  fiLp 


I 


It  should  be  pointed  out  that  in  one  sense  there  is  really  no  problem 
here.  The  spaces  Lg  and  Ig  are  isomorphic;  --  an  analog  filter  and  its 
digital  image  under  ^  are  Just  two  names  for  the  same  abstract  object. 

Having  defined  the  effect  of  u  on  filters,  we  should  .now  like 
to  do  the  same  for  the  Fourier  and  z-transfrnas.  Tnis  can  be  done  in 
an  equally  natural  way.  Suppose  A  is  an  analog  filter,  a  bounded 
linear  operator  on  the  space  L.,  of  einalog  signals.  The  analog  signal 
space  is  mapped  by  the  Fourier  transform  operator,  say^(  ),  into  a 
new  space  Lp ,  the  space  of  Fourier  transforms.  Ihe  Fourier  transform 
of  the  operator  A,  denoted  by  (A.),  will  then  be  defined  as  an  oper¬ 
ator  on  this  transform  space  so  that  if  maps  f  to  g,  then  ^  (A)  maps 
F(s)  to  G(s).  Analogously  to  (1-17)  above,  we  require 


‘J(A)  = 


(1-19) 
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where  )  denotes  the  Fourier  transform  of  an  analog  signal  as  veil 

as  an  analog  filter.  Croing  through  the  sane  calculation  that  we  per¬ 
formed  for  u,  we  find  that  the  Fourier  transform  preserves  the  norm  of 
a  filter: 

||’3f{A)||  =  ||A||  .  (1-20) 

Similarly,  we  define  the  z-transform  of  a  digital  filter  A  by 

(1-21) 

where  )  denotes  the  z-transform  of  a  digital  signal  or  filter. 

Again,  the  norms  of  filters  are  preser/ed: 

ll'J(4)ll  =  Hell  (1-22) 

V/e  have  now  generalized  u  so  that  it  pertains  to  filters  as 
well  as  to  signals,  and  we  have  defined  the  transforms  of  filters. 

Thus,  a  diagram  analogous  to  Figure  1  can  be  drawn  I'or  filters,  and  this 
is  shown  in  Figure  2.  The  connection  between  the  Fourier  transforms  of 
analog  filters  and  the  z- transforms  of  digital  filters  is  well  defined 
by  the  three  legs  of  the  diagram,  but  nothing  more  than  that  can  be 
said  at  this  time. 

9*  Some  Familiar  Classes  of  Filters 

In  this  section  we  will  show  how  the  preceding  theory  of 
filters  applies  to  many  situations  that  are  commonly  encountered  in 
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engineering.  For  instance,  time- invariant  filtering  is  usually  expressed 
by  convolution  in  the  time  domain  and  by  multiplication  in  the  transform 
domain.  Such  time- invariant  filtering  is  described  in  the  analog  case  by 
the  fol loving  theorem: 

Theorem  7»  Let  a(t)  be  a  measurable  function  satisfying 

00 

■> 

|a(t)  1  dt  <  00  ,  (1-23) 

-00 

That  is,  let  a(t)  belong  to  Li(-oo,oo).  Let  the  operator  A  be  defined 
by  the  following  convolution  integral; 

00 

Af(t)  =  f(T)a(t-T)  dT  .  (1-24) 

-00 

Then  A  is  an  analog  filter  with  norm 

00 

11;.  1 1  <  la(t)l  dt 

V. 

-00 

rurthermore,  the  Fourier  transform  of  the  operator  A  is  mult  jlicp.tion 
by  the  function  A(s),  the  Fourier  transform  of  a(t). 

The  proof  of  this  theorem  is  a  direct  consequence  of  Schwarz's  inequal¬ 
ity  and  can  be  found  in  detail  in  Titchnarsh,^  section  3-13' 

This  theorem  applies  to  any  linear  time-invariant  filter  whose 
impulse  response  satisfies  (1-23)-  Thus,  any  stable  RLC  network  Is  an 
analog  filter.  Theorem  7  can  also  apply  to  the  case  where  A  is  the 
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identity  operator  Af  =  f,  provided  we  are  willing  to  adiiit  the  delta 
function  as  a  sifting  function  satisfying  (I-23)*  It  is  hardly 
necessary  -o  introduce  distributions  or  other  generalized  functions 
on  this  account,  however,  since  the  identity  operator  is  clearly  a 
bounded  linear  operator  in  its  own  right. 

The  following  theorem  for  tine- invariant  digital  filters  can 
be  obtained  in  exactly  the  same  way  as  Theorem  7j 


Theorem  8.  Let  [^]  be  a  sequence  of  complex  numbers  satisfying 

00 

y  <  oo  ,  (1-25) 

.-J 

n=-oo 


and  let  the  operator  /y  be  defined  on  the  space  of  digital  signals  by 
the  following  convolution  sura: 


oo 

i  (in’  =  I  k  Vl 

i=-oo 


(1-26) 


Then  A  is  a  digital  filter  with  norm 

oo 

llill<  I  Kl  . 

n--oo 

Furthermore,  the  z-transform  of  the  operator  A  is  multiplication  by 
A(z),  the  z-transform  of  the  sequence  {aj^]. 

Now  consider  the  case  where  the  digital  filter  A  in  Theorem  8 
is  the  image  under  u  of  the  analog  filter  of  Theorem  7-  Since  the 
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Fourier  transform  of  Af  is  A(8)F(s),  the  2- transform  of  the  digital 
signal  u(Af)  is 


-/L- A  (IziL.')  F  A(z)F(z)  , 

z+1  ^  z+1  ^  2+1  ' 

Therefore 

A(z)  =  A(^-|i^  .  (1-27) 

Thus,  the  transforms  of  filters  which  are  equivalent  under  the  iso¬ 
morphism  u  are  related  by  a  simple  change  of  variable.  Hais  observa¬ 
tion  will  be  useful  when  we  consider  the  approxim..tion  problem  for 
digital  signals. 

By  reversing  the  roles  of  the  time  and  transform  domains  in 
Theorems  7  and  8  we  come  to  consider  the  operation  of  multiplication  by 
bounded  time  functions,  or,  in  electrical  engineering  terms,  amplitude 
modulation.  More  specifically,  we  have  the  following  pair  of  theorems: 

Theorem  9»  Let  a(t)  be  a  bounded  measurable  function  of  time,  and  let 
the  operator  A  be  defined  on  the  class  of  analog  signals  by  multipli¬ 
cation; 

Af(t)  =  a(t)f(t) 

Then  A  is  an  analog  filter  with  norm 

I  1a|  1  <  sup  la(t)  1 
all  t 


(1-28) 
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Iheorem  10.  Let  {n^^}  be  a  bounded  sequence  of  complex  numbers  and  let 
the  operator  A  be  defined  on  the  class  of  digital  signals  by  multipli¬ 
cation 

4  (In)  =  (Siln)  (^-=9) 


Then  A  is  a  digital  filter  with  norm: 


||a||  <  sup  |anl  . 
all  n 


The  proofs  follow  immediately  from  the  relations 


00 

la(t)f(t)12  dt  < 

-00 


00 

dt 

-OO 


end 


f 


I  l^nfnl^  < 

n=-oo 


n--oo 


V/hen  the  transforms  of  a(t)  and  {aj,}  are  in  Li(-ao,oo)  and  Lj(0,2j;/T), 
respectively,  it  can  be  shown  that  the  transforms  of  these  multiplica¬ 
tion  operators  are  convolution  operators  on  the  ju>-axis  or  unit  circle. 
This  representation  is  not  important  for  us,  however. 

It  is  now  easy  to  see  that  bounded  linear  operators  are  not  in 
general  commutative.  That  is,  if  /.  and  B  are  two  bounded  linear  oper¬ 
ators,  then  it  is  not  necessarily  true  that  B(Af)  -  A(3f).  Take,  for 
example,  the  case  where  I.  is  multiplication  by  u(t),  and  3  is  an  RLC 
filter.  I.  and  B  do  commute,  however,  in  the  special  cases  when  A  and 
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B  are  both  multiplications  as  in  Theorems  9  or  10*  In  general,  we  have 
the  following  results  concerning  combinations  of  operators:^ 

Theorem  11.  If  A  and  B  are  bounded  linear  operators,  then  A+B  and  AB 
are  also  bounded  linear  operators,  and 

<  ||A||  >||B||  , 

and 

||AB|!  <  ||A|1  .  ||B||  . 

10.  A  General  Matrix  Representation  for  Filters 

V/e  have  seen  in  the  last  section  how  certain  classes  of  filters 
can  be  represented  in  the  time  domain  by  convolution  with  time- invariant 
weighting  functions  or  by  multiplication.  It  would  be  desirable,  how¬ 
ever,  to  have  a  representation  valid  for  any  bounded  linear  operator. 
Such  a  representation  can  be  constructed  in  the  sane  way  that  matrices 
can  be  constructed  from  linear  operators  on  a  finite  dimensional  vector 
space;  -  that  is,  by  examining  the  effect  of  an  operator  on  a  set  of 
elements  which  forms  a  basis.  Tnus,  if  A  is  a  linear  operator  in  a 
finite  dimensional  vector  space  of  dimension  n,  and  if  [ej ,  e- , . . . , e^} 
is  a  basis,  ve  can  assemble  the  following  array  of  equations: 

Ae^  =  a^jCj  +  aj^ep  +  ...  +  ajne^ 

Ae^  =  a^je^  +  ^2^2  ■*■••• 

•  •  • 

^‘■n  “  °^nn®n 


(I-30) 
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In  this  vay,  every  linear  operator  is  associated  with  a  unique  nxn 
matrix  Conversely,  every  rocn  matrix  detemines  a  linear  operator; 

for  if  X  is  a  vector  with  components  {x»  ,x^, . . .  ,Xn); 

n  n  n 

/vx  =  /.  7  XiCi  =  ^  •  (^-31) 

i=l  j  =  l  \=1 

This  procedure  allows  us  to  characterize  bounded  linear  oper¬ 
ators  in  the  infinite  dimensional  case,  provided  ve  impose  an  appropri¬ 
ate  condition  on  the  elements  of  the  matrices  involved: 

Definition:  Tno  infinite  matrix  is  said  to  be  bounded  if 

-  i.j-’i,j=_oo  - 

for  some  constant  M  we  have 

£,  3^  ^  S  §. 

jsT'r  i=-p  "  i=‘^p  j=-r 

for  any  numbers  x_p,x_p^^, . . .  ,Xq,Xi  , . . .  ,x^  and  y.j.,y.j.+i^  • '  •  ^ 

V/e  then  have  the  following  result,  which  is  proved  in  ^ddiiezer 
and  Glazman:^ 

r  1^0 

Theorem  12.  Let  leij^  be  an  orthonormal  basis  for  the  Hilbert  space 
-  i=-oo 

H.  Then  every  bounded  linear  operator  determines  a  unique  bounded  in¬ 
finite  matrix  [o-ij]  by 

00 

^  f  ^  ~  . . .  ,-1,0, 1,2, . . .  .  (1-33) 

j=-oo 
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Conversely,  every  bounded  infinite  matrix  determines  a  bounded  linear 
operator  in  the  following  way:  If  feH  has  the  orthonormal  expansion 


put 


CO 

f  =  y  , 

'_>j 

i=-oo 


00  oo 

"  I  C  I  ' 

J=-oo  i=-cx) 


For  a  fixed  basis,  we  write  A  'N/{aj^j]  whenever  the  bounded 
linear  operator  A  admits  the  bounded  infinite  matrix  representation 
{aji^j}.  In  analogy  with  the  finite  dimensional  case,  it  can  be  shown 
that  if  A  ''-'{aij}  and  B  'v{bj[j},  then 


A+B (aij+bij}  ,  (1-35) 

and 

00 

y  -  (1-36) 

l:=-oo 

where  BA(f)  =  B(A(f)).  We  have  thus  constructed  a  matrix-mechanical 
representation  of  signal  filters,  very  much  like  that  employed  in 
quantum  mechanics.  Sometimes  it  wi.ll  be  convenient  to  think  of  a 
filter  as  being  disconnected  from  both  zhe  input  and  the  output  for 
negative  time.  In  this  case  we  need  only  consider  the  lower  right 
quandrant  of  the  matrix; 

The  interpretation  of  this  raatrix  representation  in  the  digital 
case  is  rather  simple,  mostly  because  the  particular  basis  we  have 
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chosen  has  on  easily  grasped  physical  significance,  Ihe  n-th  basis 
element  for  the  digital  signal  space  is  just 

=  [ •  •  •  >0^  •  •  • )  >  =  . . . , -1,0,1,2, . . . , 

where  the  one  is  in  the  n-th  place.  This  is  the  image  under  u  cf  the 

n-th  orthonormal  Laguerre  function  X^it).  Thus,  the  z- transform  of 

{e  }  is  and  the  z- transform  of  a  digital  signal,  written  as  a 
~n 

power  series  in  z,  is  a  formal  representation  of  its  orthorormal  ex¬ 
pansion.  The  element  in  the  matrix  representation  of  a  digital 

filter  A  then  corresponds  to  the  output  of  the  filter  a«  time  j  when 
has  been  applied.  If  any  signal  {f^}  is  applied,  cne  output  sig¬ 
nal  will  be 

00 

■i  (£n)  =  {  ^  h  '‘in}  <  (I-3T) 

i=-oo 

by  (1-34).  In  the  time-invariemt  case,  we  can  write 
®ij  "  ®J-i 

emd  then  the  effect  of  a  digital  filter  can  be  described  by  the  familiar 
convolution  formula 

00 

A  [i!n^  "  {  2j  ^  ^-ij  *  {l"38, 

i=-oo 

This  can  be  written  in  the  z- transform  domain  as 


A  {f^}  =  A(z)F(z) 


(:-39) 


3^ 


^ere 


GO 

A(z)  -  I  z-1 

i --00 


(I-  '^0) 


is  the  z- transform  of  the  digital  filter  A.  Frcxn  (1-33)  we  see  thet 

00 

y  ki  P  <  00  > 

l  =  -00 

BO  that  the  velghtlng  sequence  (a^]  is  in  1^  and  (1-40)  is  the  z-trans' 
form  of  a  digital  signal. 

A  common  type  of  digital  filter  is  the  so-called  recursive  or 
autoregressive  filter  defined  by; 


N 


M 


Mini  =  {  E  \i(n-k)-  I  ii(n-k)} 


k=o 


:c=i 


This  filter  produces  each  output  by  talcing  a  linear  combinatiun  of  past 
outputs,  and  past  and  present  inputs.  It  is  time- invariant  and  its  z- 
transform  is 


A(z) 


y 


Since  the  one-sided  sequence  {o^}  is  in  Ig,  A(z)  mu-^t  be  analytic  oua- 


side  tne  unit  circle. 
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11.  Relationship  to  the  Weighting  Function  Representation  in  the 
Analog  Case _ 

The  mapping  p  does  not  affect  the  matrix  representation  of  a 
filter,  since  it  maps  basis  elements  into  basis  elements.  Thus,  if  A 
is  an  analog  filter  and  ^  is  its  digital  image  under  p,  we  have 


and 


00 

AXi(t)  *  ^  aijXj(t)  , 

J=-oo 

00 

j=-oo 


The  Interpretation  of  the  matrix  representation  {a^  ]  is  somewhat  more 
difficult  in  the  analog  case,  however,  because  we  do  not  usually  thlnlc 
of  an  analog  signal  as  being  represented  by  the  coefficients  in  its 
Leiguerre  expansion;  while  we  do  think  of  a  digital  signal  as  being  made 
up  of  its  values  at  the  discrete  observation  times.  Furthermore,  we 
usually  think  of  an  analog  filter  as  being  defined  in  terms  of  the  con¬ 
volution  integral 

00 


/vf(t)  = 


f(T)a(t,'r)  dT  , 


(1-43) 


-00 

where  a(t)  is  some  weighting  function.  We  are  thus  faced  wi  the 
problem  of  relating  this  representation  to  the  matrix  representation 
{aj[j}.  When  a  is  the  identity  operator,  a(t)  is  the  delta  function,  so 
we  see  immediately  that  we  cannot  expect  a(t)  to  be  a  proper  function 
in  the  general  case.  We  can  proceed  formally,  however,  in  the  follor' 
ing  manner:  Letting 
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GO 

f(t:  =  ^  fiXi(t)  , 

i=^QO 

we  can  write 

OO  00  oo 

Af(t)  =  ^  fiAXi(t)  *  ^  ^  ^ 

1—00  i=-oo  J=-oo 

since  the  bounded  operator  is  ccMitlnuoua.  Replacing  by 

00 

f(T)Xi(T)  dT  , 

-00 

and  interchanging  the  orders  of  integration  and  sunnnation,  we  have 

CO 

'*  CO  00 

Af(t)  »  f(T)  ^  (T)Xj(t)l  dT 

i  -oo  j=-oo 

-00 

We  therefore  have  the  fomal  equivalence 

no  oo 

i^-oo  j=-uo 

The  problem  is:  if  A  is  a  bounded  operator,  what  kind  of  function  will 
(1-44)  be.  We  would  expect,  in  general,  that  a(t,T)  will  be  a  distrihi- 
tion,  but  a  theorem  to  this  effect  does  not  exist  in  the  mathematical 
literature  and  is  certainly  not  obvious.  We  will  therefore  content 
ourselves  with  the  formal  connection  between  the  bounded  u^xurix 
and  the  weighting  function  a(t,T)  given  by  (1-44). 
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The  foraula  Inverse  to  (I- 1*4)  can  be  derived  as  follows:  The 
effect  of  A  on  Xi(t)  is 


AXi(t) 


oo 

Xi(T)a(t,T)  dT  = 

00 


00 


z 


aij\j(t)  . 


(1-45) 


Therefore, 


00 


00 

n 


Xj(t) 


c 

•00 


Xi(T)a{t,T)  dx  dt 


-00 


(1-46) 


Here  we  have  assumed  that  (1-45)  is  in  Lg  and  hence  can  be  expanded  in 
a  Laguerre  series. 

ifnea  a^j  =  the  digital  filter  A  will  have  as  its  z- 

trensforra  multiplication  by  A(z).  Hence,  the  Fourier  transform  of  the 

ard  therefore 

a(t,T)  =  a(t-T).  Conversely,  if  a(t,x)  =  e(t-T),  the  Fourier  transform 

of  A  will  be  multiplication  by  A(s),  and  hence  the  z-traxisform  of  A  will 

be  multiplication  by  A  ^ This  Implies  that  a^  .  =  VJe  see 

V  2+1  y  J 

that  an  analog  or  a  digital  filter  will  be  time- invariant  when  eind  only 

when  a . ,  can  be  written  a .  . . 
ij  J-i 

Tliose  'dme- invariant  filters  which  are  physically  realizable 
are  of  great  i.aportance  in  many  fields.  A  time- invariant  analog 
filter  A  is  called  realizable  if  .'S  =  0  for  t  <  0  whenever  f  =  0  for 
t  <  0.  Similarly,  a  time- invariant  digital  filter  A  is  called  realiz¬ 
able  if  A{^}  =  0  for  a  <  0  ^enever  (f^)  =  0  for  n  <  0.  It  is  an 
important  property  of  the  mapping  n  that  it  always  matches  time- invariant 


(1 4*3 

-r - 

1-S  . 
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realizable  filters  with  time- invariant  realizable  filters.  To  see  this 
suppose  first  of  all  that  A  is  a  time- invariant  realizable  analog  filte'. 
Let  (f^)  be  any  digital  signal  for  which  (^)  =  0  for  n  <  0.  Then  its 
analog  image  f(t)  is  such  that  f(t)  =  0  for  t  <  0.  Ilhus  Af  =  0  when  t 
is  negative,  and  this  implies  that  =  0  for  n  <  0.  Tuis  shows  that 

A  is  a  realizable  digital  filter.  Ihe  same  argument  works  the  other  way, 
eind  this  establishes 

Theorem  13.  The  mapping  n  for  filters  always  matches  tin'-- invariant 
realizable  filters  with  time- invariant  realizable  filters. 

A  time- invariant  digital  filter  is  realizable  if  end  only  if  a^^j  "  ®-J-i 
=  0  \dien  1  >  J.  Hence,  it  follows  that  a  time- invariant  analog  filter 
is  realizable  when  and  only  when  a^j  =  aj_i  =  0  if  i  >  j. 

We  can  thus  characterize  all  time- invariant  realizable  filters 
by  upper  triangular  infinite  matrices  of  the  form 


^■o 


0  ®0 


.  0  0  Sq  aj  Sg  . 

•  • 
0  0  0  a^  Sj 


0  0  0  0  Sq 


(l-it?) 


Thus,  a  tine- invariant  analog  filter  is  determined  canpletely  by  .  ;s 
response  to  any  Xj^(t);  Just  as  a  time- invariant  digital  filter  is 
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determined  completely  by  its  response  to  any  e  *  {...  ,0,0, 1,0,0, . 
It  follows  also  that  the  response  of  a  realizable  time- invariant  analog 
filter  to  Xi(t)  will  have  no  Xj  components  when  J  <  i.  That  is,  the 
output  vector  in  response  to  Xj^(t)  is  orthogonal  to  Xj(t)  ^en  j  <  i. 
For  example,  if  we  apply  XgCt)  to  a  realizable  time- invariant  analog 
filter  A,  we  would  expect 


-00 

Using  Perseval's  relation  (1-4)  and  writing  A^Cs)  for  the  Laplace  trans¬ 
form  of  Xj^(t),  this  becomes 

A8(s)A(s)Ai(-s)  ds  =  0  , 

-joo 

which  is  indeed  true,  since  the  integrand  is  analytic  in  the  ri3ht-half 
s-plane. 

In  the  time-varying  case,  on  the  other  hand,  m  does  not  pre¬ 
serve  realizability.  To  see  this,  consider  the  bounded  operator  'rlth 
the  matrix 

^is  =  1 

^ij  “  0  ,  otherwise 

This  corresponds  to  the  digital  filter  which  delays  fj  one  unit  but  has 
zero  output  at  other  times.  Thus,  A  is  a  realizable  digital  filter. 

The  analog  filter  A,  however,  is  given  by  (1-43)  and  (1-44). 


4o 


oo 

0 

Af(t)  =  Xa(t)  j  f(T)>.,  (t)  dT  , 

I 

C 

-OO 

and  Is  not  realizable. 

To  illustrate  the  relationship  between  the  matrix  and  the 
weighting  function  representation  of  a  time- invariant  analog  filter,  we 
will  take  up  as  an  example  the  all- pass  function 


\  _  1-s 

/»(s)  -  '■  " '  • 

1+s 

Multiplication  by  A(6)  in  the  transform  domain  defines  a  bounded  oper¬ 
ator;  in  fact,  the  analog  filter  A  leaves  the  energy  constant  of  any 
signal  invariant.  The  digital  filter  /.(z)  corresponding  to  A(s)  is  z"'', 
a  unit  delay.  Hence,  the  matrix  representation  of  these  operators  is 

=  1  if  j  =  1  .  1 
=  0  otherwise 

According  to  (I-iA),  then,  the  weighting  function  a(t,T)  is 

oo 

a(t,T)=  2^  •  (1-46) 

i--oo 

Assume,  without  real  loss  of  generality,  that  t  >  0  and  t  >  0.  (1-48) 

then  becomes 

oo 

a(t,T)  =1  y  Xi(T)Xj^.^j^(t)  .  (1-49) 


i=l 


We  now  need  the  following  two  Identities,  which  are  given  in  Head  and 

Wilson:^3 

00 

)  ^s(^)^s(“)  “  6(t-T)  ,  (I-50) 

s-i 


I 


00 

=  -  Y  '^2  \n+i(t-T) 

S=1 


.(I-51) 


These  identities  are  very  useful  for  putting  a(t,T)  in  closed  form  when 
the  filter  is  time- invariant.  Putting  n  =  0  in  (I-5I),  we  get  finally 


a(t,T)  =  a(t-T)  =  -  6(t-T)  +  u(t-T) 


This  checks  with  the  inverse  Laplace  transform  of  A(s) 
can  be  checked  similarly:  the  integral 


(I-U6) 

1+s 


00 

r> 


I  Xj^(T)a(t,T)  dT 


-00 


is  equal  to  and  hence 


00 


'■1.1  = 


00 

r> 


V' 

-00 


X4(T)a(t,T)  dT  dt 


■00 


is  1  when  J  =  i  +  1,  and  zero  otherwise. 

It  should  be  noted  that  the  complexity  of  the  representation 
of  analog  filters  as  compared  with  digital  filters  is  reflected  in  the 
identification  and  synthesis  problems.  To  identify  a  time- varying 
digital  filter,  one  need  only  apply  signals  [e^}  and  read  the  coefficients 


a^j  from  the  output.  Synthesis  involves  only  the  setting  of  coefficient 
in  a  digital  computation  program.  In  the  analog  filter  case,  however, 
there  is  no  such  natural  and  convenient  basis.  If  we  apply  Xj^(t)  as  a 
testing  function,  we  must  then  resolve  the  output  into  a  Laguerre  series 
and  use  a  formula  like  (1-44)  to  arrive  at  a  weighting  function.  Even 
then,  we  are  left  with  the  problem  of  realizing  a(t,T)  in  the  general 
case. 

Still,  it  is  a  significant  and  not  widely  mentioned  fact  that 
any  analog  filter,  even  a  time-var3'’ing  one,  can  be  characterized  by  an 
infinite  matrix  of  numbers.  More  importaiit,  this  fact  has  not  been  put 
to  full  use  in  the  development  of  identification  techniques.  In  fact, 
most  identification  techniques,  whether  they  are  based  on  a  weighting 
function  representation,  ?  differential  equa^Lc..  model,  a  time- varying 
transfer  function  model,  or  an  orthogonal  filter  expansion,  usually 
assume  that  the  system  is  stationary'  over  short  measurement  segments. 

The  matrix  representation,  on  the  other  hand,  is  a  very  general  rep¬ 
resentation,  valid  even  for  fast  varying  systems. 

12.  Optimization  Problems  for  Systems  with  Deterministic  signals 

We  are  now  in  a  position  to  see  how  some  optimization  problems 
can  be  solved  simultaneously  for  analog  and  digital  signals.  Svppose, 
for  example,  that  a  certain  one-sided  noise  signal  n(t),  and  that  we 
are  required  to  filter  out  the  noise  with  a  stable,  realizable  time- 
invariant  filter  H  whose  Laplace  transform  is,  say,  H(s).  If  we 
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adopt  a  least- mean- square  error  criterion,  we  require  that 


oo 

r 

1  (r-H(r+n))^  dt  =  min  .  (1-52) 


/»8  described  by  Cheuig,^  this  can  be  transformed  by  Parseval's  relation 


to  the  requiretient 


2:rj 


[R-H(R+N)][R-H(R+N)]  ds  =  min. 


-Joo 


(1-53) 


where  R,  H,  and  N  are  functions  of  s,  and  the  overscore  indicates  that 
s  is  replaced  by  -s.  It  can  be  then  shown,  using  an  adaptation  of  the 
calculus  of  variations,  that  the  realizable  solution  for  H(s),  say  Hq(s), 
is  given  by 


Y  ^  Y  -'lKP 


(1-54) 


where 


YY  =  (R+N)(R+N)  , 


eind  Y  has  only  left- half  plane  poles  and  zeros,  and  Y  has  only  right- 
half  plane  poles  and  zeros.  The  notation  (  indicates  that  a 

partial  fraction  expansion  is  made  and  only  the  terms  involvi.ng  left- 
half  plane  poles  are  retained. 

The  fact  that  a  least- mean- square  error  criterion  is  used  means 
that  the  optimization  criterion  (1-52)  can  be  expressed  in  the  axiomatic 
framework  of  Hilbert  space.  Thus,  in  the  Hilbert  space  Lo(-oo,oo), 

(I- 52)  becomes 


l|r-H(r+n)l|  =  min  .  (1-55) 

If  we  now  apply  the  mapping  ^  to  the  signal  r-H(r*-n),  we  see  that 

1  |r-H(r^n)  1  I  =  ]  |u(r-H(rt-n) )  1 1  -  !  |r-H(r+n)  |  j  ,  (1-56) 

since  \x  preserves  norm.  Hence  is  a  solution  to  the  optimisation 
problem 

1  Ir-H(r+n)  !  1  ^  min  .  (1-57) 

Furthermore,  since  u  matches  one-sided  analog  signals  with  one-sided 
digital  signals,  and  since  matches  realizable  time- invariant  analog 
filters  with  realizable  time- invariant  digital  filters,  we  see  that  ^ 
is  the  solution  to  a  digital  optimization  problem  that  is  completely 
analogous  to  the  original  analog  problem.  In  addition,  the  general 
solution  (I- 5^)  can  be  translated  into  digital  terms  by  replacing  the 
left-half  plane  by  the  unit  circle  in  an  appropriate  way.  Thus, 


where 


^(z) 


(R^^N)R 

Y 


YY  =  (R+N)(R+N) 


(1-58) 


Now  R,  and  N  are  functions  of  z;  the  overscore  indicates  that  z  is 
replaced  by  z”^ ;  Y  and  Y  have  poles  and  zeros  inside  and  outside  of  the 
unit  circle,  respectively;  and  the  notation  [  ]jjj  indicates  that  only 
the  terms  in  a  partial  fraction  expansion  with  poles  inside  the  unit 


circle  have  been  retained 


kf> 


In  other  optimization  problems  we  may  wish  to  minimize  the  norm 
of  some  error  signal  while  ):eeping  the  norm  of  some  other  system  signal 
within  a  certain  range.  In  a  feedback  control  system,  for  instance,  we 
may  want  to  minimize  the  norm  of  the  error  with  the  constraint  that  the 
norm  of  the  input  to  the  plant  be  less  than  or  equal  to  some  predetermined 
number.  Using  Lagrange's  method  of  undetermined  multipliers,  this  prob¬ 
lem  can  be  reduced  to  the  problem  of  minimizing  a  quantity  of  the  form 

lleir  -  k"  lllIP  ,  (1-59) 

where  e  is  an  error  signal,  i  is  some  energy  limited  signal,  and  both 
e  and  i  depend  on  an  undetermined  filter  function  H.  Again,  if  Hq(s) 
is  the  time-invariaiat  realizable  solution  to  such  an  analog  problem, 
then  §o(z)  will  be  the  time- invariant  realizable  solution  to  the  analo¬ 
gous  digital  problem. 

It  is  aLmost  always  important  to  us  that  the  solution  to  an 
optimization  problem  be  realizable,  but  we  may  want  to  allow  as  a  solu¬ 
tion  a  time-varying  filter.  Iftifortunately,  the  isomorphism  u  does  net 
necessarily  raaten  realizable  time-varj'lng  analog  filters  with  realizable 
time- varying  digital  filters.  V/e  thus  cannot  show  that  optimization 
problems  which  allow  time-vaiT-ing  solutions  are  equivalent  in  the 
analog  and  digital  cases.  Furthermore,  since  any  known  isomorphisms 
involve  orthogonal  expansions  of  the  analog  signals  over  semi- infinite 
or  infinite  raiiges  of  time,  it  appears  that  an  isomorphism  between 
Lp(-oo,oo)  and  Ig  which  preserves  the  realizability  of  filters  cannot 
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be  constructed.  Thus,  in  order  to  show  the  “qui valence  of  an  analog 
with  a  digital  optimization  problem,  we  require  that  the  allowed  class 
of  filters,  say  be  invariant  under  a  particular  isomorphism. 

Another  way  of  looking  at  the  problem  is  to  say  that  we  are  really  de¬ 
manding  that  the  entire  optimization  problem  be  expressible  in  the 
terms  of  abstract  Hilbert  space.  Thus,  when  the  class  is  the  class  of 
time- invariant  realizable  filters,  we  can  characterize  in  abstract 
Hilbert  space  as  the  class  of  all  bounded  linear  operators  A  for  which 
aj^j  =  a^_j^  and  a^_^  =  0  for  i  >  J. 

We  can  therefore  state  that  any  optimization  problem  which  can 
be  expressed  solely  in  terms  of  abstract  Hilbert  space  can  be  solved 
simultaneously  for  analog  and  digital  systems.  In  particular  we  can 
state: 

Theorem  I4.  Let  u  be  an  isomorphism  between  Lp(-oo,oo)  and  1^.  ’^ther, 
let  the  following  optimization  pi’oblem  be  posed  in  the  analog  signal 
space  lig (-00,00):  Find  analog  filters  Hi,H2,...,Hn  which  minimize  some 
function  of  some  norms  in  a  given  analog  signal  transmission  system  and 

is 

invariant  under  u,  the  corresponding  digital  problem  is  equivalent  to 
the  original  analog  problem  in  the  sense  that  if  one  can  be  solved,  so 
can  the  other. 

As  we  have  seen,  the  case  where  is  the  class  of  time- invariant 
realizable  filters,  and  v  is  n,  is  an  important  application  of  this 
result.  In  this  situation  we  have  the  following  correspondences: 


which  are  in  a  class  of  filters 


'Phen  if  the  class  of  filters 
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13.  Random  Sifrnals  and  Statistical  Optimization  Problems 

\Vhile  the  consideration  of  sycteniii  with  deterministic  signals 
is  important  for  many  theoretical  and  practical  reasons,  it  is  more 
often  the  case  tliat  the  design  engineer  Icnovs  only  the  statistical 
properties  of  the  input  and  disturbing  signals.  For  this  reasor,  t,i>e 
design  of  systems  on  a  statistical  basis  has  become  increasingly  impor¬ 
tant  in  recent  years.  In  this  section  we  shall  show  that  the  idea  of 
linking  continuous  theory  with  discrete  theory  can  be  extended  to  a 
broad  class  of  random  phenomena;  namely,  stationary,  ergodic  processes 
with  well-behaved  correlation  functions  and  spectra. 

Because  a  complete  axiomatization  of  random  processes  is  a  very 
complex  affair,  we  will  simplify  matters  by  approaching  the  subject 
through  the  correlation  function.  This  is  not  nearly  so  restrictive  as 
it  might  first  appear,  because  physical  stochastic  processes  almost 
always  have  correlation  functions  that  are  of  exponential  order,  and 
their  spectra  are  almost  always  bounded  and  continuous.  For  a  more 
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complete  dlscuasion  of  random  Bipnal  theory  and  cenerallzod  harmonic 

Q  TO 

analysis,  the  reader  is  referred  to  V/iener.  *  Accordingly,  we  assume 
that  random  signals  arc  stationery,  ergodic,  and  have  zero  mean.  If 
x(t)  end  y(t)  are  two  such  random  signals,  we  assume  further  that  the 
cross- correlation  function 


^xyC^)  -  E[x(t)y(t+T)] 


(1-60) 


dies  down  exponentially  with  increasing  |t|.  Hie  notation  E[  ]  means 
"ensemble  average  of."  Since  the  processes  are  ergodic,  (l-60)  can  be 
expressed  as  a  time  average; 


lim 
T  -  00 


T 

f 

x(t)y(t+T)  dt 


(1-61) 


o 

Now  let  Xm(t)  and  y,j(t)  be  the  sane  signals  as  x(t)  and  y(t)  for 
0  <  t  <  T,  but  zero  outside  of  this  range;  and  let  X,p(6)  and  Yrr,(s)  be 
their  Le.place  transforms.  The  cross- spectral- density  function  is  then 
defined  by 


lim 
T  -  00 


E[:'  (-s)Yjs)] 

1  X 


(1-62) 


It  is  a  classical  result  of  generalized  harmonic  analysis,  called 
Wiener's  theorem,  that  and  transform  pairs: 

00 


> 


00 


^^xy('t)e'St  dt 


(1-63) 


and 


iy^y(z)C^^  dS 


(I-6J0 


-ioo 


In  the  important  case  when  x  =  y,  the  variance  oC  x  is  given  by  (1-64): 


E[x^]  =  ^^(0)  = 


V„y(s)  ds 


(1-65) 


As  we  might  expect,  a  parallel  theory  exists  In  the  di^A^al 
case.  Here,  if  x^  and  are  two  discrete  stationary,  and  ergodic 
random  processes,  the  cross- correlation  function  is  defined  by 


^y{n)  =  E[xixi+n] 


(1-66) 


Again  with  ergcdicity,  we  have 


4y(n)  =  lim  y  x^Xi^.^}  . 

N  -  00 


(1-5?) 


The  cross- spectrr.l-density  is  a  function  of  z,  defined  ly 


5  (z)  =  lim  i  ElX-^(z-M%(z)]  , 


N  -  00 


(1-68) 


where  Xj^(z)  and  Yjj(z)  are  the  z- transforms  of  signals  which  coincide 
with  Xj^  and  yj^  for  0  <  i  <  N,  and  which  are  zero  outside  this  range. 
lis  in  the  analog  case,  ji(j^y(n)  and  ^^y^z)  are  transform  pairs: 
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?xy(2) 


N 

11m  y  ^3jy(n)z‘*’' 

N  -  00  ^  „ 
n^-N 


(I-l'9) 


vhlch  exists  on  the  unit  circle  If  we  assume  that  the  correlation 
function  dies  off  exponentially  as  n  -  oo;  and 


4y(z) 


2nJ  , 

|z|-l 


SjCyCz) 


.n  dz 


(1-70) 


The  variance  of  the  signal  in  analogy  with  (I-65},  is 


=  -2^  9  -T 


(1-71) 


z  j=l 


!nie  parallel  with  the  deterministic  case  Is  so  strong  when  the 
reuadom  theory  is  put  in  the  above  form,  that  the  intixaduction  of  the 
mapping  u  presents  no  problem.  Consider  (1-62),  for  example.  If  we 
map  the  transformed  analog  signal  X^(s)  to  ['/2/(  z+1 )  ]X^  we 

should  map  Xt(-s)  to 


+  1  ^  z-^  +  1 


Similarly,  Y(p(s)  shoxild  map  to  ['/2/(z+l)]  Yip  ^  accordance 

with  (I- 68),  we  define  the  mapping  ^  by 


u;  ^  (t)  -  5  (s)  -• - 

xy  (z+1)®  z+1 


'xyCiir)  ■  -  V’ 


(1-72) 
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nie  reverse  mapping  goes 

icy(“)  “ixy(»)  "  ®xy  • 


We  have  thus  defined  a  mapping  which  maps  analog  to  digital  cross 
correlation  functions.  Hie  Important  Invariants  under  ^  are  the 
quantities 

^y(O)  =  E[x(t)y(t)]  , 
and 


which  correspond  to  the  Inner  product  In  the  deterministic  case, 
verify  that  these  are  preserved  under  |i,  put  t  =  0  in  (1-64): 

joo 


4y(0) 


2itJ 


^xy(®)  ds 


-Joo 


Z"*  1 

If  we  now  sake  the  change  of  variable  s  *  -  ve  get 


(1-73) 


To 


Since  all  of  the  steps  In  (1-72)  and  (1-73)  are  reversible  and  give 
unique  results,  the  mapping  p  is  clearly  one-to-one  and  onto. 


59 


or 


o 

(0  ■  -=-arctan  co  . 

T 

Suppose  now  liiat  we  are  given  seme  periodic  function  of  o), 
C(aj)say,  that  is  to  be  the  desired  characteristic  (magnitude,  phase 
angle,  real  or  imaginary  part)  of  a  digital  filter.  C  (^-|-arctan  co^ 
will  then  be  the  corresponding  characteristic  for  an  analog  filter. 

We  can  then  approximate  C  Q-^arctan  oij  as  an  analog  filter  charac¬ 
teristic,  using  any  one  of  the  many  procedures  available  for  analog 
filters.  We  thus  arrive  at  a  rational  function  of  s,  say  A(s).  Then 

A(  z)  «•  A  (^1?:  will  be  a  digital  filter  whose  characteristic  approocl- 
-  ^  z+1 

mates  the  desired  one.  Since  the  left-half  s-plane  is  mapped  Inside  tho 
\anlt  circle  in  the  z-plane,  stable  poles  of  the  analog  filter  A(s)  will 
beceme  stable  poles  of  the  digital  filter  A(  z) . 

B 

Loosely  speaking,  we  have  taken  the  interval  lo)]  <  jt/T  and 
stretched  it  out;  done  o\ir  apxnroxlmatlon  for  an  analog  filter;  and 
then  squeezed  the  co-zxis  back  into  the  original  Interval.  Although  the 
to-axis  is  compressed,  many  of  the  widely  used  approximation  criteria, 
such  as  equal  ripple,  maximal  flatness,  etc.,  carry  over  directly  to 
the  digital  filter  case.  If  an  analog  filter  A(s)  has  magnitude  M(a)), 
phase  angle  $((o),  real  part  R(u)),  and  Imaginary  part  I(u));  then  the 
corresponding  digital  filter  A(z)  will  have  magnitude  M(tan  (dr/2), 
phase  angle  5(tan  aff/2)  in  la)|  <  x/T,  real  part  R(tan  <dr/2) ,  eind 
Imaginary  part  I(tan  o^/2) . 
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As  an  Illustrative  example,  suppose  we  wish  to  approxlaiate 
the  ideal  lov-pass  characteristic  shown  as  a  dashed  line  In  Figure  U. 
We  have  taken  the  cutoff  frequency  to  te  at  co  ■  k/2T,  one-half  the 
Nyqulst  frequency.  The  analog  filter  A(s)  should  therefore,  by  (11-3)^ 
have  an  Ideal  cutoff  at 

03  »  tan  03^/2  ■  1  . 

Let  us  now  use  for  A(s)  a  third-order  maximally  flat  Ritterworth  low- 
pass  fUter^^  with  unit  cutoff  frequency: 


A(s)  ■ - = - 

s®  +  2s^  +  28  +  1 

When  we  let  s  »  this  becomes  the  digital  filter 

z+1 

A(z)  .  ,1  3Z-W  3Z-V  z-3 

"  3  +  Z”2 


(n-5) 


whose  normalized  magnitude  Is  shown  plotted  as  curve  B  in  Figure  4. 

A(  z)  is  now  a  maximally  flat  digital  filter.  Its  response  Is  zero  at 
the  Nyqulst  frequency  o)  »  jt/T,  this  point  corresponding  to  Infinite 
frequencies  for  the  analog  filter  F(s).  The  filter  A(z)  can  be  Imple¬ 
mented  In  a  hand  or  machine  computation  according  to  ( 1-^1) ,  Section  10. 
Thus,  If  and  g^,  respectively,  are  the  input  and  output  digital  signals; 


81 


^(fl  +  Jfi-I  +  Jfi-Z  +  fi-3  -  gi-s)  . 


(II-6) 
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A  typical  application  of  such  a  snoothlng  operation  would  be  to  remove 
high  frequency  noise  prior  to  halving  the  number  of  data  points. 

As  a  more  elaborate  example  of  a  smoothing  routine,  suppose 
we  wish  a  low-pass  filter  with  a  sharp  cutoff  at  one-quarter  the  Nyqulst 
frequency,  co  «  jt/^T.  This  corresponds  imder  the  mapping  p.  to  the 
frequency 


0)  a  tan  aff/2  *  tan  «/8  «  0. 41^*2  .  (II”7) 

Suppose  further  that  we  desire  the  digital  filter  to  have  equal  ripple 
In  the  pass  band.  We  might  then  start  off  with  the  fourth-order 
Tchebycheff  filter  having  abotrt  10f5  ripple  ( «  l/5) ,  and  with  a  cutoff 
frequency  at  o)  =  II 

A(s)  a - i -  .  (II-8) 

s^  +  1.054s3  +  1.555S^  +  0.8506s  +  0.50G2 

If  we  substitute  (s/0.4l42)  for  s,  we  get 


s'*  +  0.1f284s^  +  0.2655s^  +  0.05905s  +  0.009011 

(II-9) 

which  has  a  cutoff  freqxjency  at  ou  a  0.4l42.  We  then  substitute 
2-1 

s  a -  to  obtain  the  desired  digital  filter: 

z+1 


Ai(z) 

zs 


b: 


1  +  4z’^  +  6z”2  +  4z’3  +  z"* 


1.760  -  4.705z'^  +  5.5272"^  -  5.225z"3  +  0.7849z"4 


( TI-IO) 
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Figure  5  shCTfs  the  normalized  nmgnitude  of  this  Tchehycheff  equal 
ripple  dlgited  filter.  If  this  filter  ;rere  used  prior  to  a  one-half 
data  reduction,  noise  at  frequencies  greater  than  half  the  Nyqulst 
frequency  vould  affect  the  resulting  signal  very  little.  If  the  power- 
spectral-denslty  of  the  resviltlng  reduced  digital  signal  were  measia^d, 
it  vould  be  desirable  to  correct  for  the  ripples  in  the  frequency  char¬ 
acteristic  of  the  filter  A(  z) .  The  design  of  a  high-pass  or  a  bandpass 

Vt 

digital  filter  follows  the  same  pattern. 

We  have  thus  seen  how  the  mapping  p.  allows  us  to  reduce  the 
approximation  problem  for  digital  filters  to  that  for  analog  filters. 
The  technique  described  allows  the  designer  of  digital  information 
processing  systems  to  deal  with  signals  in  the  frequency  domain  in 
much  the  same  way  that  the  comniunl  cat  ions  engineer  deals  with  analog 
signals. 

5.  Comparison  with  Fourier  Series  Techniques 

19 

GulUemln  ^  has  suggested  the  use  of  Fourier  series  for  the 
approximation  of  magnitude  characteristics  of  analog  filters,  ttis 
approximation  procedure  consists  of  using  the  mapping  p  to  convert  the 
desired  chai*acterlstlc  to  one  that  is  a  periodic  function  of  frequency, 
using  a  truncated  Fourier  series  in  co  to  approximate  this,  and  then 
inverting  the  transformation  p  to  give  a  rational  function  of  o.  Since 
we  deal  directly  with  periodic  magnitude  characteristics  as  a  function 
of  a>,  we  can  use  Fourier  series  directly.  Thus  the  use  of  Fourier 
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series  is  a  natural  choice  for  the  design  of  digital  filters,  and. 
GulUemln  reversed  ovir  program  and  used  It  for  the  design  of  analog 
filters . 

Suppose,  then,  that  we  are  given  the  desired  magnitude  charac“ 
terlstlc  M(a))  of  sooie  digital  filter.  Since  this  Is  an  even  function 
of  <D  with  period  2r/t,  we  can  approximate  It  In  a  least  mean-square - 
error  sense  with  the  truncated  Fourier  series 


where 


K 

M(a))  -  ^  S 

n— K 


Si 


T 

2n 


n/T 

M(a))  e*^*^  do) 
«/T 


( ii-n) 


(11-12) 


The  realizable  digital  filter 

2k  K 

A(z)  -  ^  Cjj.j^  z’°  -  z"^  Z  Si  (11-13) 

n»o  n»-K 


w’lU  Uien  have  a  magnitude  characteristic  which  approximates  M(^ , 
because  when  z  »  e*^^ 


|A(z)  1 


n— K 


Si 


a:  M(a)) 


(11-14) 
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This  technique  Is  particularly  valuable  for  two  reasons.  First, 
since  the  series  (II-U)  Is  a  cosine  series,  the  only  phase  distortion 
Is  that  caused  by  the  delay  factor  z"^.  Thus,  If  the  delay  of  KT  Is 
tolerable,  there  Is  essentially  no  phase  distortion.  Second,  these  fil¬ 
ters  are  polyncmlals  In  z"^  and  have  no  denonlnator.  Therefore,  their 
lioplementatlon 

61  =  C-K^l  +  c.K+lfi.l  +  . . .  +  c^f  1.2K  ^  n-15) 

does  not  require  the  storage  of  outputs.  This  leads  to  programs  which 
can  be  easily  effected  by  simple  special  purpose  computers  and  which 
require  a  relatively  small  amount  of  storage  capacity. 

On  the  other  hand,  the  fact  that  these  filters  are  polynomials 
In  z"^  means  that  there  Is  a  loss  of  several  degrees  of  freedom.  This 
usually  leads  to  magnitude  characteristics  that  have  the  ripple  and  over¬ 
shoot  characteristic  of  Foxu'ler  series  approximations.  Looking  at  this 
problem  In  another  way,  we  can  consider  these  Fourier  series  filters, 
or  any  other  polynomial  filters,  as  power  series  approximations  to  ra¬ 
tional  functions,  since  by  (I-40)  emy  time -invariant  dlgltel  filter  can 
be  written  as  an  infinite  series: 

00 

A(z)  «  ^  a^z’^  .  (n-16) 

l**-00 

We  would  therefore  expect  a  finite  polynomial  In  z"^  to  have  more  ripple 
and  overshoot  than  a  properly  designed  rational  function,  whose  power 


series  does  not  terminate . 
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To  Illustrate  these  points^  suppose  we  again  want  a  lov-pass 
digital  filter  with  a  cutoff  frequency  at  one -half  the  Nyqulst  frequency. 
M(u))  is  the  ideal  characteristic  shown  as  a  dashed  line  in  Flgiire  6. 
Equation  (11-12)  then  yields  the  following  Fourier  coefficients: 


Cn  ■  c-n  -  ^ 


1/2  , 


njt 


0 


n  «  0 


f  n  “  l/5;5> • •  • 


(11-17) 


The  normalized  magnitude  characteristics  of  the  first  three  of  the  re¬ 
sulting  digital  filters  are  plotted  in  Figure  6: 


Curve  a:  A(z)  -  -i-  +  -iLz*!  +  -L-z’®  (11-18) 

*  n  2  jt 

Curve  B:  A(  z)  »  +  — z“^  +  +  — z’**  - 

■  5Jt  «  2  «  3n 


Curve  C:  A(z)  *■  -  -^z’^  +  -i-z”**  +  -i-z“®  +  -^z“®  -  -J^z"®  +  -^z"^  . 

“  5«  »t  2  n  3jr  5« 

We  note  the  ripple  and  overshoot  described  above.  One  way  to  alleviate 

20  21 

this  difficulty  would  be  to  use  Fejer  means  *  of  the  coefficients  Cjj. 

This  would  produce  smooth  approximations  ,  but  at  the  expense  of  having 
a  slower  cutoff  and  poorer  rejection  in  the  stop-band.  In  any  event,  If 
we  need  a  digital  filter  with  a  magnitude  characteristic  that  is  both 
close  to  Ideal  and  smooth,  we  must  use  either  polynomials  of  very  high 
order  or  rational  functions. 
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Ccorparlflon  with  z»Tranflforms  of  Axialo/;  Filters 

Vfe  take  up  now  another  approximation  method|  one  that  at  first 
appears  natxxral,  but  Is  actually  not  very  premising.  Suppose  that 
instead  of  taking  the  ^-transform  of  an  analog  filter,  A(8),  we  take 
the  ordinary  z-transform.  In  this  case,  the  resulting  digital  filter 
is  given  by 

00 

y  A(Ja)  +  Jn2it/t)  .  (11-19) 

a  T  L, 

n«-oo 

Typically,  A(s)  would  be  designed  so  that  it  approximates  the  desired 
digital  filter  characteristic  for  loj]  <  n/T,  and  is  small  in  magnitude 
outside  this  range.  If  A{s)  then  has  all  its  poles  Inside  the  left-half 
plane,  A(z)  will  be  a  stable  digital  filter  with  approximately  the  de- 

«s 

sired  charewJteristic.  The  main  difficulty  with  this  method  is  the  addi¬ 
tion  of  unwanted  terms  in  (11-19)  due  to  aliasing  of  the  filter  function. 

To  use  this  idea,  we  must  start  with  analog  filters  which  have  carefully 
tailored  characteristics  with  sharp  cutoffs  and  good  rejection  In  stop- 
bands,  and  all  this  leads  to  high  order  filters. 

Furthermore,  finding  the  z-transform  of  a  high  order  filter 

involves  a  great  deal  more  work  than  Just  letting  s  ■  -"■“j'  ;  and  ^en 

z+1 

we  are  done,  ve  must  recalculate  the  magnitude  or  phase  characteristic 
of  the  result  to  assess  the  errors  Introduced  by  the  eliasing  of  the 
original  characteristic.  All  in  all,  the  u-transform  is  much  better 
suited  for  the  purpose  of  con'/ertlng  analog  filters  into  useful  digital 


filters. 
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To  illustrate  the  above,  consider  again  the  third-order  maxi¬ 
mally  flat  Butterworth  lov-pass  filter  that  we  considered  In  Section  2: 


A(s) 


_ 1 _ 

s^  +  2s^  +  2s  +  1 


The  z-transform  of  this  filter  is 


(11-19) 


0.3703z“^  0.1546z~^ _ 

1  -  0.3981z'^  +  0.2k7kz~^  -  0.04321 


(11-20) 


The  normalized  magnitude  characteristic  of  this  digital  filter  is  plotted 
as  curve  A  in  Figure  4,  which  also  shows  the  magnitude  characteristic  of 
the  p, -transformed  filter.  Because  of  the  relatively  high  cutoff  fre¬ 
quency  of  A(s)  (one-half  the  Nyquist  frequency),  and  because  of  the  low 
order  of  A(s),  the  effects  of  the  aliasing  of  the  filter  characteristic 
are  quite  pronounced  —  the  cutoff  is  not  sliarp  and  the  rejection  is  poor. 

In  summary,  tlie  mapping  p.  converts  the  approximation  problem 
for  digital  filters  to  the  approximation  problem  for  analog  filters. 

This  latter  problem  has  received  a  great  deal  of  attention  over  the  past 
fifty  years,  and  we  are  fortunate  to  be  able  to  use  it  to  otir  p^u^)oses. 


5.  Building  .Anedog  Filters  with  Digital  Ccmputers 

Vfe  conclude  Part  II  with  a  discussion  of  the  possibility  of 
constructing  an  analog  filter  from  a  sampler,  a  digital  filter,  and  a 
data  reconstruction  device.  Such  a  system  would  probably  be  Implemented 
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In  recLL  time  using  a  digital  computer.  The  advantages  of  using  a  digital 
computer  as  an  analog  filter  are  the  flexibility,  accuracy,  and  stability 
which  can  be  readily  obtained,  and  which  ore  practically  Impossible  to 
achieve  with  analog  hardware.  The  coefficients  In  a  digital  computer 
program  can  be  set  to  a  high  degree  of  accuracy,  can  be  changed  very 
fast,  and  are  not  subject  to  unwanted  variation  with  temperature  or  age. 
Furthermore,  with  the  use  of  pulse -code  modulation  for  the  low  noise 
transmission  of  signals  over  large  distances,  the  avalJabllity  of  signals 
already  In  digital  form  can  make  It  more  feasible  to  filter  In  real  time 
with  a  digital  con^ter.  Ultimately,  however,  whether  such  a  scheme  Is 
practical  depends  on  the  state  of  computer  technology. 

Suppose  then  that  we  sample  an  analog  signal  f(t),  pass  the 
resulting  digital  signal  through  a  digital  filter  A(z),  and  then  recon¬ 
struct  an  analog  signal  with  a  data  reconstruction  circuit  H(  s) ,  The 
Laplace  transform  of  the  output  signal  is 

G(s)  =  A(e®'^)H(s)F*(s)  ,  (II-21) 

where  F*(b)  Is  the  Laplace  transform  of  the  sampled  input.  We  can  thus 
write  a  transfer  function  with  respect  to  the  sampled  input: 

»  A(  e®*^)  H(  s)  .  ( 11-22) 

We  assme  now  that  we  have  sampled  at  a  frequency  at  least  twice  as 
great  as  the  bandwidth  of  f(t).  Then,  in  the  range  loo]  <  jt/T,  the 
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transfer  function  (11-22)  represents  the  effect  of  the  system  on  the 
original  signed,  and  outside  this  range  represents  spurious  harmonics 
of  the  input  signal  caused  by  Imperfect  data  reconstruction.  These 
upper  sidebands  can  be  removed  with  a  simple  low-pass  analog  post-filter 
having  a  cutoff  frequency  near  the  Nyquist  frequency. 

As  an  example,  suppose  that  H(s)  is  a  zero-order  hold: 


sT 


(11-23) 


iH(ja))  1 


sin  air/2 
dr/2 


lH(ja3)  I  has  its  first  zero  at  twice  the  Nyquist  frequency,  and  has  lobes 

of  apixreclable  magnitude  well-outside  the  range  |ai|  <  n/T.  Hence,  the 

overadl  transfer  function  (11-22)  will  have  spurious  responses  at  high 

frequencies  unless  these  are  filtered  out.  Suppose  now,  as  an  example, 

that  we  use  the  Tchebycheff  filter  (II-IO)  of  Section  2  as  our  digital 

filter  A(  z) .  The  normalized  magnitude  of  the  resulting  digital  filter- 

hold  combination  is  shown  in  Figure  7*  Ne  note  that  the  shape  of  the 

digital  filter  A(z)  is  slightly  distorted  in  the  pass-band  by  being 
I  sin  dT/2  , 

multiplied  by  | - —  I*  necessary,  the  magnitude  characteristic 

of  A(z)  can  be  compensated  to  correct  for  this  distortion. 

It  is  interesting  to  note  that  the  filtering  chtiracteristic  of 
our  final  system  can  be  changed  as  fast  as  the  coefficients  in  the  digital 
computer  program  can  be  changed.  If  we  used  bandpass  digital  filters, 
for  example,  we  might  then  be  able  to  use  the  system  to  replace  a  bank 
of  fixed  filters  or  a  frequency  sweeping  system. 
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PART  III:  APPLICATIONS  TO  Pg/ER  SPECTRUM  MEASURHffiHT 


1.  Introduction 

The  concept  of  power-spectral-density  has  become  an  Important 
tool  for  the  analysis  and  synthesis  of  many  t;,-pes  of  physical  systems. 

As  a  result,  there  is  a  pressing  need  for  ways  to  estimate  the  power- 
spectral-density  of  a  signal  from  a  finite  record  of  that  signal. 
Originally,  analog  methods  provided  the  only  practical  way  to  do  this. 
These  methods  usually  involve  the  selection  of  a  narrow  band  of  fre¬ 
quencies  with  a  bandpass  analog  filter,  and  then  a  measurement  of  the 
power  density  of  the  signal  in  this  bend.  Too  wide  a  pass-band  results 
in  an  averaging  of  the  spectral  density  over  an  excessively  wide  range 
of  frequencies,  with  a  resulting  decrease  in  resolution;  v/hile  too 
narrow  a  pass-band  results  in  excessive  statistical  fluctuations  of  the 
estimates.  In  195^^  Chang^^  derived  an  expression  for  the  optimum  band¬ 
width  for  the  spectrum  analyzer  and  showed  that  the  optimum  shape  for 
the  spectrum  analyzer  was  semicircular. 

In  recent  years,  when  high  speed  digital  computers  became 
available,  methods  for  spectrum  analysis  based  on  equally  spaced  samples 
of  the  signal  of  interest  were  developed.  These  methods  were  at  first 
divorced  from  the  concept  of  a  bandpass  filter,  xmtll  the  concept  of  a 
spectral  window  was  Introduced.  Still,  the  connection  between  the  analog 
and  digital  methods  of  spectmm  estimation  has  remained  obscure.  One 
goal  of  this  part  will  be  the  clarification  of  this  connection.  He 


71 


begin  with  a  review  of  the  methods  for  spectrum  analysis  of  equaiiy 


sjjaced  data,  based  mostly  on  the  work  of  Blackman,  Tukey,  and  Press. 

First  of  all,  if  a  randoa  signal  is  sampled,  the  sampled  i^ower 
spectrum  of  the  resulting  digital  signal  is  related  to  the  original 
spectrum  by 

00 


24,?5,26 


( III-l) 


n=-oo 


where  T  is  the  sampling  interval.  V/e  see  Immediately  that  we  must 
sample  at  a  rate  fast  enough  to  reduce  imdesirable  aliasing  of  the  origi¬ 
nal  spectrum.  Otherwise,  the  spectrum  we  measure,  ^ 

accurate  reflection  of  the  spectrum  of  the  original  signal. 

Assuming  that  we  have  sampled  fast  enough,  and  have  prefiltered 
the  original  analog  signal  to  reduce  high  frequency  noise  if  necessary, 
we  can  compute  estimates  of  the  autocorrelation  function.  \Je  assume 
throughout  this  part  that  we  have  observed  samples  Xj^,X2, . .  .Xjj  of  the 
original  signal  x(t),  and  that  N  is  so  large  that  we  can  neglect  end 
effects.  Th\is  we  compute  the  (iir»-l)  mean  lagged  products 

H-|k| 

I  ^A+lkl  '  .  (III-2) 

i=l 


These  fj^  are  unbiased  estimates  of  the  autocorrelation  ftmction  ^jj-r(k): 


lim  fjj 
N  -«  00 


(III-5) 
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since  the  power  spectrum  is  given  by  (I-69)s 


N 

-  lla  ^  » 

“  -  “  kt-N 

we  are  led  to  the  estimate 


(III-4) 


k»-m 


(III-5 


where  Zq  =  and  ojq  Is  the  frequency  of  Interest.  This  estimate  Is 

known  as  the  perlodogram.  These  estimates  are  statistically  unstable 
because  they  give  equal  weights  to  all  the  fj^,  while  the  fj^  for  larger 
k  are  much  less  reliable.  This  suggests  weighting  the  sum  (II-5)  in  the 
following  manner  1 


where 


m 


XX 


k=-m 


(III-6) 


w  =  w  , 
k  -k 


The  expected  value  of  this  estimate  is 


k=-m 


(Cont'd.) 
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Cont'd. 


I'^i  ,  k»-m 

lzl»l 

«/T 

^jQj(ai)  MioirOiQ)  dm  , 


_L 

2it 


(III~7) 


-«/T 


where 

m 

VKz)  -  ^  (Ill-e) 

ka-m 


Is  a  weighting  function  which  determines  an  estimate  of  $^(a>)  and  is 

called  the  spectral  window.  3y  a  convenient  abuse  of  notation  we  write 

SjOfCoj)  instead  of  problem  of  choosing  a  good  spectral 

window  has  received  much  attention.  A  good  evalua*  '.on  of  many  spectral 

27 

windows  can  be  found  In  Grenander  and  Rosenblatt,  ' 


2.  A  Class  of  \7indow8  Generated  by  Dlpcital  Filters 

If  we  now  try  to  mimic  the  analog  method  for  spectrum  analysis, 
using  digital  filters  instead  of  analog  filters,  we  are  led  to  a  speclcLL 
class  of  estimates  involving  a  special  class  of  spectral  windows.  Suppose 
then  that  we  design  a  bandpass  digital  filter  that  is  timed  to  the  parti¬ 
cular  frequency  cu^,  say  D(z) .  Let  us  pass  the  digital  signal  xi,...,Xjj 
through  this  filter  to  obtain  an  output  sequence  yi,...,yjj.  The  power 
density  of  this  output  signal  is  then  the  average  energy: 
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A  N 

1-1 

Tiie  expected  value  of  this  estimate  Is 
E^xx^^o)  -  E(yf)  -  ^(0) 


1 

2rtJ 


(h  D(z)D(7-1) 


n/T 


2 

|D(a>)  I  ?xx(“) 


C III-IO) 


so  that  these  estimates  correspond  to  the  weighting  function 

\/(cu-<Uq)  -  |d(cu)  .  (III-U) 

Thus,  this  special  class  of  estimates  has  the  desirable  property  of 
having  a  weighting  function  that  Is  always  positive.  This  means  that 
no  matter  >diat  the  shape  of  the  original  spectrum,  the  estimates  will 
always  be  positive,  a  situation  that  is  not  always  true  for  the  more 
general  estimates  using  windows  U(a)-aj^) .  We  therefore  have  eliminated 
the  problem  of  negative  power  leaking  through  a  side  lobe  of  the  i/eightlng 
function. 

In  general,  the  Implementation  of  the  estimate  (III- 9)  will 
necessitate  running  the  digital  signal  through  the  digital  filter 
D(z)  for  each  frequency  of  Interest.  This  is  a  decided  disadvantage. 
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because  of  the  long  time  that  this  would  take  on  a  coc5)Uter.  In  the 
special  case  when  D(z)  is  a  pol^canial,  howver,  we  can  compute  the 
estimates  directly  from  the  fj^.  To  see  this,  write 


1 

2«J 


CP 


D(z)D(z"^) 


N 


dz 

z 


where  we  define 


( 111-12) 


X(z) 


+  XgZ”^  +  XgZ"^  + 


+  Xjjz 


-N 


( III-13) 


Assmlng  that 

K 

D(z)  a  ^ 
kal 


(III-12)  becomes 


A 

5xxC%) 


K 

^  <i]5djefic-i» 

k,f=o 


( m-i4) 


which  is  Just  as  easy  a  quantity  to  calculate  on  a  computer  as  ( III-6) . 
The  coefficients  dj,  will,  of  covurse,  be  different  for  each  frequency  of 
Interest . 

V/hether  we  use  this  last  method  and  restrict  D(z)  to  be  a  poly¬ 
nomial  or  we  use  a  rational  function  of  z  and  run  the  signal  through  the 
filter  for  each  measurement,  we  can  now  use  tne  approximation  methods 
discussed  in  Part  2  to  design  spectral  vdndows.  It  is  very  easy  to  use 
different  windows  for  different  parts  of  the  spectrum,  for  we  have 
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ccmplete  control  over  the  shape  of  the  window  at  all  tljnes.  We  have 
thus  seen  ho^r  the  measurement  of  power-si)ectral-density  for  discrete 
signals  can  be  thought  of  in  terms  of  filtering  and  energy  measurements, 
just  as  in  the  analog  case. 
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We  see  from  this  that  the  bias  error  is  due  entirely  to  the  fact  that 

the  welghtlnc  function  is  not  a  6 -function.  For  this  reason  we  may  coll 

25 

It  the  "blurring "  error,  after  Chang. 

The  variance  is  somewhat  more  difficult  to  calculate.  From 
(II-9)  we  have  first 


A  ** 

E[v(a3)f=-^  y  ECygygl  .  ( III-20) 

IF 

n,m=l 

In  order  to  evaluate  these  fourth-order  moments,  we  now  assume  tliat  the 
original  signal  has  a  normal  probability  distribution  function.  With 
this  assumption,  the  digital  signal  y^  is  also  nc>rmally  distributed,  and 
using  the  characteristic  fvinction  for  the  we  get 

ECygyJ]  =  ^^(0)  +  2  .  (III-21) 


Thus, 


N 


ELv(a3o)]  =  ^(0)  + 


( III-22) 


n,m=l 


Also,  we  have 


.r> 


[E.-.(a>„)r  =  li^yCO)  , 


( III-25) 


so  that 

N 

variance  =  )  p^(m-n) 

n,m-l 


( III-24J 
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This  can  be  put  in  terms  of  the  power-spectral-density  of  the  by 
using  formula  (I-70): 


°  2nj 


CD  fyy(z) 


n  dz 


T 


2n 


Ul=i 

Hence,  we  can  \rrite 


•«/T 


4yy(Ci>)  dO)  .  (111-25) 


Jt/T  Jt/T 


variance  =  — ~(T/2n)‘ 

ir 


-n/T  -n/T 


/-- 

U^l 


^'yy(COl)jyy(C^)  dCUldUV,  . 

( III-26 


Using  the  identity 


N 

')  g-jy(a3i-afe)T 


sin^'  -^(cui-ct)T 
sin^  _^(u)i-ct:)T 


(III-27) 


this  becomes 


variance  =  (T/2rt)^ 

II- 


«/T  n/T 

sin*"  _il_( 0)1 -0^2)7 
_ 2 _ 

sin^  _^(a)i-afe)T 

n/T  -n/T 


^'yy(c^>l)';yy(a^)  dcoidcjqo  . 

(III-26) 


The  inner  integral  in  this  expression  is  laiown  as  Fejor’s  integral,  and 
is  discussed  in  Carslaw,^^  and  in  Titclunarsh.^^  The  essential  result  is 


that 


lim  Ein^  NcUT 

N  00  - N  6(cu)  5  (III-29) 

sin"  -i-  oiT  ^ 

O 


79 


Tliat  is,  the  Fejer  kernel  tends  to  a  £ -function  vlth  increasing  N. 

S-*’  in  our  case  N  is  very  large  (usually  larger  than  50  or  100  for  a 
meu  ■  ^ful  spectral  analysis),  we  can  write  as  a  good  approximation 


Thus  the  variance  is  inversely  proportional  to  N,  which  is  in  agreement 

27 

with  Grenander  and  Rosenblatt,  '  who  used  a  different  derivat^'on  that 
applied  to  spectral  windows  that  are  not  necessarily  generated  by  digi¬ 
tal  filters.  Furthermore,  if  we  normalise  by  the  square  of  the  area 
under  the  window! 


(III-51) 


the  variance  is  inversely  proportional  to  the  length  of  the  record  NT, 
which  agrees  with  the  analog  case.  Me  thus  have  deri'/ed  an  expression 
for  the  mean-square-error  of  the  digital  filter  estimates: 


n/T 

jD(a))  I.  clau 

-n/T  ( III-52) 
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4.  The  Optlxam  Digital  Filter 


Our  program  now  Is  to  find  the  digital  filter  shape  that  mlnl- 

23 

mlzes  this  nean-square-error,  thus  following  the  derivation  that  Chang 
presented  for  analog  filters.  Accordingly,  we  represent  the  digital 
filter  characteristic  lD(a))  1^  by  U®(fl)/A),  where  0  =  %  is  the 

center  frequency  to  which  D(a))  is  tuned,  and  A  is  sene  kind  of  bandwidth 
such  that  U®(fi)/A)  is  small  for  |nl  >  A.  We  thus  focus  our  attention  on 
only  one  main  lobe  of  the  digital  filter  characteristic,  at  to  =  We 
assume  also  that  the  filter  D(z)  is  sharply  tuned  to  (%,  so  thit  we  can 
write  to  a  good  approximation 


variance 


4-  -Ir  W 


This  becomes  in  our  new  notation: 


variance  = 


2 

T 

N 

2k 

2 

T 

N 

2k 

0 tat ion: 

4 

T 

N 

2k 

k/T 


.^/T 


lD(a))  1^ 


«/T 


-«/t 


iD(a))l'‘ds>  .  (III-55) 


00 

r> 


5^  (CD  ) 
xx^  o' 


i/*(f)/A)  an 


(111-34) 


-00 


To  express  the  bias  simply,  we  expand  the  spectral  density  in  a 

power  series  about 
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+  -|“  ^/^(oj(a>-ajo)^  + 


'xx 


(III-3:;) 


Assxaning  that  the  area  under  the  filter  characteristic  is  one,  or, 
equivalently,  that  our  estimates  are  adjusted  by  dividing  by  the  area 
under  the  filter  characteristic,  the  bias  term  (III-19)  becomes 


bias 


«/T 

lD(a))  I  (oj-o)^)  du) 

-«/T 


$  (oj  ) 
XX'  O'' 


00 


-00 


L/^(ft/A)  dft 


(III-36) 


where  we  have  also  assumed  that  U^(n/A)  is  an  even  function;  that  our 
filter  has  a  symmetrical  magnitude  characteristic  about  the  resonance 
frequency. 

b/e  seek  to  minimize  the  normalized  mean-square-error,  given  by 


oo 

(T/2jt)2[2  J  U^(fi/A) 

-oo 


2 


dn  1 


(III-37) 


For  convenience  of  notation,  we  now  define  the  follo^dug  integrals 
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00 


J 

•00 


U^(n/A)  d(fi/A) 


00 

J  »  J  u^(fi/A)(n/A)^  d(n/A) 


-CD 


00 


K  = 


i;*(n/A)  d(n/A) 


u 

-00 


(111*58) 


(III-39) 


( III-40) 


V7ith  these  notations,  the  normalized  mean-square -error  becomes: 


e 


s 

nozm 


12 


A^ 


5^(%) 


( III-41) 


which  Is  exactly  the  same  as  Equation  (25)  In  Chang's  paper, which  was 
derived  for  an  analog  filter  Instead  of  a  dlgltaJL  filter.  Hence,  the 
rest  of  the  derivation  Is  Identical  to  that  for  the  anedog  case,  and  we 
are  done. 

Thus,  the  optimum  bandwidth,  obtained  by  differentiating  (III-41) 
with  respect  to  A  and  setting  the  resxdt  equal  to  zero.  Is 


r— 

1 

K 

V  NT  ^ 

( III-U2) 


We  have  therefore  shown  that  the  optimum  bandwidth  Is  Inversely  propor¬ 
tional  to  the  length  of  the  record  available  for  spectral  analysis. 

Tills  optimum  bandwidth  was  derived  by  Grenander  and  Hosenblatt  for  two 
specific  windows;  we  have  here  shown  that  It  holds  In  general  for  windows 
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generated  by  digital  filters. 

The  nornialized  nean-sqxiare-crror  in  the  case  that  the  bandwidth 
is  chosen  optimally  is 


■norm 


(2n/rrr)‘® 


— 


(111-43) 


Since 


/ 


■norm 


j5. 

2 


K 


‘  [»;,(%)  C-|r)T^  ( ni-'*'*) 


we  can  define  the  error  coefficient 
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( III-45) 


which  is  a  quantitative  measure  of  how  small  an  expected  error  can  be 
achieved  with  filter  charecteristics  of  different  shapes.  As  Chang 
shows,  the  optimum  shape  for  the  function  U  is  given  by 


U(n/A)  =0  ,  for  |r.|  >  A 

U®(fl/A)  -  A{1  -  (n/&)^)  ,  for  |n|  <  A  . 


( 111-46) 


This  being  obtained  by  setting  the  first  order  variation  of  Kg  with 
respect  to  U  equal  to  zero.  This  semlcirciilar  filter  shape  gives 
Kg  =  0.G6.  The  shape  of  the  filter  is  actually  not  too  critical,  pro¬ 
viding  that  the  bandwidth  has  been  chosen  well,  and  providing  that  the 


8^ 


side  lobes  of  the  digital  filter  characteristic  in  the  region  |fll  >  A 
are  small.  Thus,  the  Ideal  rectangular  filter  shape 


U(fi/A)  =0  ,  for  |n|  >  A 
U(n/A)  =1  ,  for  |n|  <  A 


( 111-^7) 


yields  Kg  =  0.60,  which  is  close  to  optimum.  Thus,  the  bandpass  trans¬ 
formations  of  a  low-pass  Butterworth  or  Tchebycheff  filter  would  serve 
well  as  spectral  windows,  although  the  Fourier  series  filters  wovild  be 
easier  to  Implement  using  ( III-14) . 


5.  Prewhltenln/?  Techniques 

Ue  thus  see  how  the  approximation  techniques  described  in  Part  2 

can  be  applied  to  the  design  of  spectral  windows.  These  approximation 

techniques  are  *l1so  especially  useful  in  prewhitening  spectra  before  the 

above  estimation  methods  are  applied.  The  idea  of  prewhitening  has  been 

26 

strongly  advocated  by  Hlackman  and  Tuloey  for  a  few  reasons,  one  of 
which  can  be  seen  by  examining  the  e:q)resslon  for  the  mean-square -error 
C III-41) .  This  error  depends  directly  on  the  second  derivative  of  the 
spectrum  at  the  measurement  point,  which  appears  in  the  bias  term.  If 
we  could  sonehow  flatten  the  spectrum  before  measurement  and  then  compen¬ 
sate  for  this  after  the  estimates  have  been  computed,  we  would  reduce 
the  bias  term  vri.thout  affecting  the  variance.  Another  advantage  of 
meastiring  an  essentially  flat  spectrum  is  that  there  is  then  little 
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possibility  of  an  unreasonable  contribution  from  a  peak  In  the  spectrum 
that  happens  to  correspond  to  a  minor  lobe  in  the  spectral  window. 

Therefore,  if  we  have  a  rough  idea  of  the  shape  of  the  spectrum 
we  are  measuring,  we  can  approximate  this  shape  with  a  digital  filter 
D( z) ,  so  that 

|D(aj)  2:  .  (III-i+8) 

We  can  then  pass  the  original  signal  through  a  digital  filter  1/D(z)  , 
producing  a  signal  \d.th  a  relatively  flat  spectrum.  Estimates  of  this 
power-spectral-denslty  are  then  con^juted  in  the  usual  way,  and  then  cor¬ 
rected  by  multiplying  by  lD(a))  |^.  The  techniques  described  in  Part  2  are 
well  suited  to  accomplish  this  prewhltenlng  in  an  ox'ganized  way. 

6.  The  Identification  of  Power  Spectrum  Parameters* 

Suppose  now  that  a  system  designer  needs  to  know  the  power- 
spectral-density  of  some  signal.  Assuming  that  he  has  an  idea  of  the 
bandwidth  of  the  signal,  he  can  obtain  samples  of  it,  calculate  the  mean 
lagged  products  fj^,  and  then  use  some  spectral  window  to  estimate  the 
spectral-density.  What  he  gets  after  this  procedure  are  estimates  at 
points  along  the  frequency  axis,  usually  equally  spaced.  If  the  results 
of  this  spectral  analysis  are  going  to  be  xxsed  for  anything  besides  a 
visual  presentation,  the  designer  will  have  to  put  this  in  some  closed 
aneLLytical  form.  One  way  to  do  this  is  sviggested  by  the  mapping  11. 

The  points  of  the  power  spectrum  can  be  transformed  by  the  mapping  p.  as 


*  Tne  results  in  the  remainder  were  reported  by  the  author  in  Reference  28. 
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In  Equations  ( 1-72)  and  1-73)  •  The  neasiurensent  points  ncs/  represent  the 
spectrum  of  an  analog  signal,  and  this  can  be  put  in  the  form  of  a  ra¬ 
tional  function  of  s  by  using  Eode's  method  of  semi-infinite  slopes.  The 
reverse  mapping  will  then  yield  a  rational  function  of  z,  which  is  a 
form  which  can  be  used  for  explicit  design.  This  procedure  leaves  much 
to  be  desired.  First  of  all,  it  Involves  two  consecutive  approximations 
and  the  accuracy  of  the  final  result  is  difficult  to  gauge.  Second,  it 
is  not  easily  mechanized  on  a  computer,  and  is  hence  ill-suited  for  real 
time  application  as  an  identification  method  for  adaptive  systems. 

It  would  therefore  be  desirable  to  have  a  method  of  measuring 
power-spectral-density  that  yields  an  analytical  form  for  the  answer. 

The  technique  of  prewhltenlng  suggests  the  foUowiTig  method  of  acccan- 
plishlng  this:  siippose  we  pass  the  signal  of  interest  through  a 
digital  filter  D(  z)  of  some  canonic  form,  and  then  adjust  the  coefficients 
of  dCz)  so  that  the  output  is  in  some  sense  most  nearly  white  noise.  Then 
we  have 

liXtu)  ^  1  ,  (111-49) 


so  that 


1 

D(z)D(z-^) 


(III-50) 


can  be  used  as  an  analytical  expression  for  the  unknown  power -spec tr al¬ 
iens  Ity. 

If  this  program  is  to  be  carried  out,  the  following  consideraticn 
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la  Importeuit;  the  most  time  coneuming,  and  hence  expensive,  step  In  a 
spectral  analysis  is  always  the  ccanputation  of  the  mean  lagged  products 
fjj.  Hence,  we  would  like  to  calcxilate  only  one  set  of  these  for  each 
spectral  analysis.  If  we  assume  that  D(z)  is  a  polynomial  in  2“^,  the 
output  mean  lagged  products  can  be  expressed  In  terms  of  the  input  mean 
lagged  products  rather  easily.  On  the  other  hand,  if  D(z)  has  even  one 
pole,  it  becomes  intractable  to  express  the  output  mean  lagged  products 
in  terms  of  those  of  the  input,  and  the  mean  lagged  products  of  the  out¬ 
put  must  be  recalculated  for  each  choice  of  coefficients  in  D(  2) ;  and 
this  becomes  impractical.  Hence,  the  procedure  outlined  is  only  practi¬ 
cal  irtien  D(z)  is  a  polynomial  in  z"^. 

At  first,  the  following  method  was  tried  on  a  computer.  D(z) 
was  assumed  to  have  the  form 

D(z)  =  1  +  az-i  +  bz-2  ,  (III-5I) 

the  input  mean  lagged  products  were  coronated,  and  the  appropriate  mean 
lagged  products  of  the  output  of  D(z)  were  computed  from  these.  The  cri¬ 
terion  for  whiteness  was  that  the  autocovarleuice  determinant  of  the  output 
signal  be  maxinun.  'By  the  method  of  steepest  descent,  the  coefficients 
a  and  b  were  found.  The  method  converged  nicely,  but  gave  good  results 
only  when  the  unknown  power  spectrum  was  of  the  appropriate  fonnt 
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Furthermore,  the  extremal  seeking  procedure  beccraes  less  reliable  when 
more  unknown  coefficients  are  Introduced. 

It  was  thf.n  found  that  the  above  problem  Is  equivalent  to  a 
well-known  problem  In  mathematical  statistics:  the  solution  of  this 

latter  problem  can  be  found  In  the  literature;  a  good  discussion  Is  given 

29 

by  Hannan,  ^  for  example.  Thus,  when  0(2)  Is  assumed  to  have  the  form 

D(z)  =  1  +  +  Oqz’^  +  ...  +  o^>z“P  ,  (III-55) 

a  completely  analytical  expression  for  the  coefficients  ot^  can  be  de¬ 
rived  In  terms  of  the  mean  lagged  products  of  the  Input  signal.  The 
method  cannot  be  extended  to  the  case  where  D(z)  has  poles,  for  essen¬ 
tially  the  same  reason  described  above.  Thus,  it  Is  the  responsibility 
of  the  experimenter  to  ensure  that  the  unknown  spectrum  can  in  fact,  be 
represented  closely  by  the  form  ( III -52) .  Seme  ways  of  getting  around 
this  problem  will  be  discussed  later.  We  now  present  the  solution  to 
the  identification  problem  described  above  when  D(z)  is  a  polynomial  In  z"^. 

7.  Statement  of  the  Problem 

We  make  the  following  assinnptlons; 

1.  N  points  of  the  signed  of  interest  are  available: 

and  N  is  large  enough  so  that  end  effects  can  be  neglected. 

2.  The  signal  Is  normally  distributed  with  zero  mean,  and  is 


stationary  and  ergodic. 


89 


3*  The  signal  h.\8  a  power-spectral-denslty  which  can  be  closely 
represented  by 

D(z)D(z*^) 

where 

D(z)  ■  1  +  a^z"^  +  Cfez“®  +  •••  +  Q^z'P 

has  all  of  Its  zeros  Inside  the  imlt  circle  In  the  z-plane. 
The  problem  Is  to  estimate  the  parameters  . . . ,0^,  and  given 

the  N  observed  points  of  the  signal. 

8.  The  Most  Likely  Estimates 

The  solution  given  here  will  be  essentially  the  sane  as  that 
29 

given  by  Hannan,  except  that  our  argument  will  be  In  terms  of  power 
spectra. 

Define  a  new  signal  y^  by  through  the  digital  filter  D(z) . 
That  Is,  put 


ClII-55) 


yi  =  xi  +  +  oyc^.g  +  •  •  •  +  Op^i-p  ,  ( III-56) 

or.  In  z-transform  notation 

Y(z)  =  D(z)  X(z) 


The  stochastic  variable  y^  Is  normally  distributed.  Furthermore,  Its 
power-spectral-denslty  Is 
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5yy(s)  -  D(z)D(z-^)  SjQjCz)  -  (III-57) 

BO  that  the  signal  Is  guassian  distributed  white  noise  with  mean  square 
2 

value  p  .  The  Joint  probability  density  function  of  the  observed  sample 

N 

p(yi>y2,-.*,yN)  “ - ItpT”  C — V"  )  • 

(2rt)N/2pN  2p2 

(III-58) 

The  maximum  likelihood  estimates  of  the  unknown  parameters; 
denoted  by  and  are  obtained  by  maximizing  this  proba¬ 

bility.  Thus  the  following  set  of  equations  must  be  solved: 


^  log  p(yi>yg^«»-iyN) 


0,  J  ^  1^2^  > .  •  ^p5 


(III-59) 


and 


3  log  p(yi;y2^«..^yii) 

38^ 


(III-60) 


When  ClII-56)  is  substituted  in  (III -58)  and  the  indicated  operations  are 
carried  out,  the  most  likely  estimates  result: 


( III-61) 


91 


and 

Ag^AA  AAA  A 

^  "  L  “i“j  +  •■•  +  Vp  ’ 

i/J=o 

where  the  fj  are  the  mean  lagged  products 

N-J 

fj  -  Y.  2  °>  •  (III-6J) 

1=1 

In  summary,  then,  the  following  computations  are  performed: 

1.  From  the  N  sample  points  of  the  signal,  the  mean  lagged  products 

are  calculated  In  accordance  with  ( III-63) . 

2.  The  pxp  matrix  (f|jL_j  jj  =  !>•••>?)  Is  formed  and  inverted. 

A 

3.  otj  (J  =  1, ...,p)  are  calculated  frcm  (III-61) 
k.  ^  Is  calculated  from  (III-62). 

These  computational  steps  are  shown  dlagramatlcally  In  Figure  6. 
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la  aayniptotlcal.Ty  nomaUy  distributed  vlth  zero  mean  and  covariance 
matrix 


£ 


N-p 


^o  4 

K  K 

4  4  4 


4-^ 


This  can  be  estimated  conveniently  by 


4“2 

>^p-3 


-1 


An 


II -p 


L*p-i 


•^O 

f. 


•p-i 


^p-2 

^p-3 


-1 


(III-65) 


(III-66) 


which  does  not  use  any  quantities  which  have  not  already'  been  calculated. 

The  distribution  of  the  estimate  ^  Is  difficult  to  calculate 
since  It  is  a  more  complicated  function  of  the  fj*s.  It  Is  easy,  however, 
to  derive  the  distribution  of 


1=1  1,J“0 


o^af 


(III-67) 


95 


and  this  will  give  sane  (optimistic)  indication  of  the  variability  of 

Ag 

P  .  With  this  in  mind  conalder  the  random  variable 

N 

'l  iyj^f  .  (111-66) 

li 

This  is  the  sum  of  squares  of  Independent,  normally  distributed  random 
variables  whose  means  are  zero  and  whose  variances  are  1.  This  random 

o  ^0 

variable  is  therefore  x  -distributed  with  N  degrees  of  freedom.  Cramer-^ 
shows  that  with  increeislng  N  the  distribution  becomes  asymptotically 
normal  with  mean  N  and  standard  deviation  Therefore,  the  random 

variable  (III-67)  is  asymptotically  normally  distributed  with  mean  and 
standard  deviation  /2/n  P^J  euid  hence  v/ 2/n  ^  can  be  used  as  a  low  esti- 
mate  of  the  standard  deviation  of  P^. 

10.  Extension  to  Spectra  with  Zeros 

As  mentioned  before,  the  assumption  that  the  unlmown  spectrum 
does  not  have  any  zeros  is  rather  restrictive,  and  the  derivation  breaks 
down  when  a  more  general  form  for  ‘'s  assumed.  Tlisre  are  some  situa¬ 

tions  when  something  can  be  done  to  extend  the  method,  and  these  will  now 
be  discussed. 

Suppose  that  the  signal  of  interest  has  an  unknown  pa.'er  spec¬ 
trum  of  the  form 


.N(z)Nrz~" 

z)  d{  z-1 


(III-69) 
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and  that  the  locations  of  the  zeros  are  known  ( at  least  approximately) . 

Then  the  signal  can  be  prefiltered  by  a  digital  filter  1/N(z)  (or  an 
equivalent  analog  filter) .  The  resultant  signal  will  then  be  of  the 
requisite  form  and  the  method  described  in  this  paper  can  be  used  to  deter- 
mine  the  pole  locations  and  0  . 

As  another  example,  suppose  that  the  signal  of  Interest,  is 
the  s\aa  of  two  independent  signals;  one  of  which  has  a  known  power  spec¬ 
trum  (such  as  ;dilte  noise  of  a  given  aa^Jlitude) ,  and  the  other  of  which 
has  only  poles  in  its  power  spectrum.  That  is,  suppose 


^  D(z)D(z-^) 


(III-70) 


The  autocorrelation  function  of  the  signal  is,  then,  the  sum  of  known  and 
unknovm  components; 

»i^(n)  =  »i^(n)  +  .  (III-71) 

The  known  components  can  be  subtracted  from  the  computed  and  the  re¬ 
sulting  mean  lagged  products 

fo  -  ‘‘i  -  •••  '  'p  -  (III-72) 

can  then  be  used  to  estimate  D(z)  end  3^. 

Other  sltmtions  suggest  themselves.  Some  pole  loca’'  may 
be  known  in  advance,  for  instance.  Tiiese  poles  can  be  removed  before 
analysis  by  a  digital  or  analog  filter.  Alternatively,  the  maximum  like¬ 
lihood  eq’aatlons  (ill -5 9)  e-’il  (TII-60)  can  be  reworked. 
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11.  An  Example 


To  demonstrate  the  method,  a  sequence  of  210  independent  normal 
random  numbers  was  passed  through  the  digital  filter  .  The 

resultant  time  series  then  had  a  power  spectnan 


(l-.5z‘^)(l-.5z) 

Thus  for  this  signal,  ansvimlng  p  »  2, 


(III-73) 


Qi  «=  -.5 

*=  0.0 

f  -  .00397  . 

Thi*ee  mean  lagged  products  were  computed: 
f^  =  .00624, 

»  .00507,  (III-75) 

fg  =  .00147, 

and  (III-6I)  used  to  give  the  estimates 

^  =  -.495 

Os  =  .0070,  (III-76) 

Ag 

6  =  .00473 

The  estimated  covariance  matrix  of  the  ^  was  calculated  from  C III-66) : 
r  .0048  -.0024*1 

L-.0024  .0048J  ,  (III-77) 
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and  it  Is  seen  that  the  are  well  within  one  standard  deviation  of  the 
OLy  The  optljnlGtically  estimated  standard  deviation  of  ^  is 

•/iTii  ^  (1/I0)p^  ,  (111-76) 

so  that  the  16  percent  actual  deviation  is  not  unreasonable. 

Figiure  9  shows  plots  of  the  actual  and  the  estimated  power  spec- 
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trum.  Also  shown  are  the  results  of  a  conventional  spectrum  analysis 
using  a  Hamming  window  and  7  mean  lagged  products.  Note  that  more  than 
twice  as  many  multiplications  were  required  by  the  conventional  method 
to  produce  similar  accuracy,  and  that  the  results  are  not  in  a  form  that 
is  suited  for  direct  use. 


slications  of  the  Identification  Method 


The  above  identification  method  is  especially  promising  for  use 
in  an  adaptive  loopj  first  because  it  can  be  implemented  in  real  time  by 
a  computer,  and  second  because  it  gives  direct  estimates  of  parameters  that 
can  characterize  a  signal  or  a  plant.  Thus,  the  following  method  of  self- 
optimizing  control  is  suggested:  a  controller  is  designed  whose  optimum 
or  near-optimum  operation  depends  on  the  Imowledge  of  the  parameters 


Q^,...,Q^  and  of  the  power  spectrum  of  some  signal  in  the  system.  From 

A  A  A^ 

a  record  of  this  signal  of  length  N  the  estimates  cx^,..,,cip  and  P  are 


periodically  calculated  by  a  digital  computer  and  used  to  adjust  the  con¬ 
troller.  In  a  partictilsa*  application,  the  choice  of  N  is  an  Important 
problem.  N  must  be  chosen  large  enough  so  that  the  estimates  of  the  power 
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spectrum  parameters  are  accurate  enough  to  be  useful.  On  the  other  hand, 

N  should  not  be  so  large  that  the  system  reacts  to  obsolete  information. 

The  identification  method  described  above  may  also  be  used  as  a 
first  step  in  a  conventional  spectral  analysis.  After  D(z)  is  estimated, 
the  original  signal  can  be  passed  through  the  filter  D(z)  and  subjected 
to  further  spectral  analysis  by  conventional  methods.  If  the  form  assumed 
for  the  spectrum  was  appropriate  the  output  will  be  nearly  white,  and  this 
procedvire  will  amount  to  an  "automatic"  prewhitening  technique  which  can 
be  used  in  conjunction  with  conventional  spectral  analysis. 

Finally,  it  might  be  mentioned  that  the  Identification  method 
can  be  used  with  the  adaptive  Information  processing  method  described  by 
Chang.^^ 

We  have  seen  in  this  part  how  the  concept  of  digital  filtering 
can  be  applied  to  the  problem  of  meas\irlng  the  power-spectral-dcnsity  of 
a  digital  signal.  We  first  showed  how  the  idea  of  bandpass  filtering  can 
be  carried  over  from  the  analog  case  to  the  digital  case  to  generate 
spectral  windows  that  always  give  positive  estimates  of  the  spectrum. 
Furthermore,  we  have  pointed  out  along  the  way  how  digital  filters  can 
be  used  to  advantage  eis  prefilters  and  postfilters  much  as  analog  filters 
are  used  for  continuous  signals.  For  these  applications,  the  approxima¬ 
tion  techniques  described  in  Part  2  are  especially  useful.  Lastly,  we 
described  a  method  of  identifying  unloiown  parameters  in  a  power  spectrum 
of  an  assumed  form;  a  method  which  is  promising  as  an  identification  pro¬ 
gram  that  can  be  incorporated  into  an  adaptive  loop. 


SUf-lM/J^Y 


Our  main  goal  has  been  to  tie  together  the  theories  of  filtering 
digital  signals  and  analog  signals.  With  the  axicaiatization  of  filtering 
and  signal  theory  in  terms  of  Hilbert  space,  we  saw  how  an  Isomorphism 
could  be  constructed  between  the  analog  and  digital  spaces  which  allowed 
us  to  transfer  many  concepts  from  one  domain  to  the  other.  The  use  of 
Hilbert  space  showed  how  the  z-transform  can  be  defined  with  much  the 
same  generality  as  the  Foxirier  transform,  and  led  to  a  definition  of  stable 
filters  that  can  be  used  in  both  the  analog  and  the  digital  cases.  We  then 
saw  how  any  such  filter,  whether  time-varying  or  not,  could  be  represented 
by  an  infinite  matrix  of  nmbers.  In  particiJ.ar,  we  saw  that  in  the  time- 
Invariant  case  the  digital  and  analog  theories  are  essentially  identlceJ.. 
Thus,  many  common  optimum-filtering  problems  can  be  solved  simultaneously 
for  analog  and  digital  signals,  both  in  the  deterministic  and  the  random 
case.  Vfc  also  looked  at  data  reduction  filters  and  their  interpretation 
in  terms  of  frequency  resjwnse. 

In  Part  2  we  showed  that  the  approximation  problem  for  tlme- 
invarlant  digital  and  anedog  filters  were  equivalent,  and  we  discussed 
seme  methods  that  were  partlcxilarly  applicable  to  the  design  of  digital 
filters  for  some  common  purposes,  such  as  prefiltering  prior  to  data  re¬ 
duction.  We  showed  in  particular  how  Fourier  series  can  be  used  to  design 
digital  filters  with  prescribed  magnitude  characteristics  that  were  poly¬ 
nomials  in  z"^,  and  hence  could  be  Implemented  econcmlcally. 
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Part  3  was  devoted  to  the  application  of  these  ideas  to  the 
problem  of  measxiring  power-spectral-density  from  digital  information. 

We  saw  in  particular  how  bandpass  digital  filters  could  be  used  as  spec¬ 
tral  windows  which  always  give  positive  estimates  of  the  power- spectral- 
density.  We  then  derived  the  optimum  bandwidth  and  the  optimum  shape 
for  such  digital  filters,  following  the  results  of  Chang^^  for  analog 
filters.  Throughout  this  discussion  we  indicated  how  the  approximation 
techniques  of  Part  2  could  be  used  effectively  In  the  processing  of  digi¬ 
tal  Information;  prewhitening  being  an  example.  \Je  then  presented  a 
method  of  identifying  iinlcnown  parameters  in  a  power  spectrum.  This  method 
results  in  an  analytlceJ.  form  for  the  spectrum,  and  is  suitable  for  a 
systematic  prewhitening  jirogram,  or  for  use  in  an  adaptive  control  loop. 
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LIST  OF  ILLUnm\TIONS 


Figure 

1  A  schematic  representation  of  the  mapping  p,  and 
to  the  varlovis  slgneJ.  spaces. 

2  A  schematic  representation  of  the  mapping  u  wid 
to  the  various  spaces  of  operators, 

sT 

3.  The  s -plane,  the  z  =  e —  plane,  and  the  cu-axls; 

4.  Curve  A  Is  the  normalized  magnitude  characteristic  of  the 
digital  filter  corresponding  to  the  z-trauisform  of  a  third-order 
Butterworth  filter.  Curve  B  is  the  normalized  magnitude  char¬ 
acteristic  of  the  digital  filter  corresponding  to  the  p -trans¬ 
form  of  the  same  analog  filter. 

5.  The  normalized  magnitude  characteristic  of  the  digital  filter 
corresponding  to  the  p- transform  of  a  fourth-order  Tchebycheff 
low-pass  filter  with  10  percent  ripple, 

6.  Fourier  series  approximations  to  an  ideal  low-pass  digital  filter 
magnitude  characteristic. 

7.  The  normalized  magnitude  characteristic  of  an  analog  lov-pass 
filter  constructed  from  a  Tchebycheff  digital  filter  and  a  zero- 
order  hold  circuit. 

8.  Estimation  of  power  spectnaa  parameters. 

9*  Ccmpeirison  of  actual  and  estimated  power  spectra. 
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Fig.  1.  A  schematic  representation  of  the  mapping 
^  and  its  relations  to  the  various  signal  spaces. 
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Fig,  2,  A  schematic  representation  of  the  mapping 
and  its  relations  to  the  various  spaces  of 
operators. 
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Fig.  5,  The  normalized  magnitude  characteristic  of  the  digital  filter 
corresponding  to  the  ^-transform  of  a  fourth-order  Tchebycheff  low- 
pass  filter  with  10%  ripple. 
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Fig,  6.  Fourier  series  approximations  to  an  ideal  low-pass 
digital  filter  niap:nitude  characteristic. 
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Fig,  7.  The  normalized  magnitude  characteristic  of  an  analog  low-pass 
filter  constructed  from  a  Tchebycheff  digital  filter  and  a  zero-order 
hold  circuit. 


Fig.  8.  Estimation  of  power  spectrum 
parameters . 


Fig.  8.  Estimation  of  power  spectrum 
parameters • 


Fig.  9.  Comparison  of  actual  and  estimated 


power  spectra. 


